Eszterházy Károly Collage

Institute of Mathematics and Informatics
A
A
RTIFICIAL
RTIFICIAL
I
I
NTELLIGENCE
NTELLIGENCE
AND
AND
ITS
ITS

T
T
EACHING
EACHING
LECTURE NOTES BY
DR. GERGELY KOVÁSZNAI
AND
DR. GÁBOR KUSPER
Table of Contents
1 Introduction ...................................................................................................................................... 4
2 The History of Artificial Intelligence ............................................................................................... 7
2.1 Early Enthusiasm, Great Expectations (Till the end of the 1960s) ........................................... 7
2.2 Disillusionment and the knowledge-based systems (till the end of the 1980s) ........................ 8
2.3 AI becomes industry (since 1980) ............................................................................................. 9
3 Problem Representation .................................................................................................................. 10
3.1 State-space representation ....................................................................................................... 10
3.2 State-space graph .................................................................................................................... 11
3.3 Examples ................................................................................................................................. 12
3.3.1 3 jugs ............................................................................................................................... 12
3.3.2 Towers of Hanoi .............................................................................................................. 15
3.3.3 8 queens ........................................................................................................................... 17
4 Problem-solving methods ............................................................................................................... 20
4.1 Non-modifiable problem-solving methods ............................................................................. 22
4.1.1 The Trial and Error method ............................................................................................. 23
4.1.2 The trial and error method with restart ............................................................................ 23
4.1.3 The hill climbing method ................................................................................................ 24
4.1.4 Hill climbing method with restart ................................................................................... 25
4.2 Backtrack search ..................................................................................................................... 25
4.2.1 Basic backtrack ............................................................................................................... 26
4.2.2 Backtrack with depth limit .............................................................................................. 29
4.2.3 Backtrack with cycle detection ....................................................................................... 31
4.2.4 The Branch and Bound algorithm ................................................................................... 33
4.3 Tree search methods ................................................................................................................ 34
4.3.1 General tree search .......................................................................................................... 35
4.3.2 Systematic tree search ..................................................................................................... 37
4.3.2.1 Breadth-first search ................................................................................................. 37
4.3.2.2 Depth-first search .................................................................................................... 38
4.3.2.3 Uniform-cost search ................................................................................................ 40
4.3.3 Heuristic tree search ........................................................................................................ 41
4.3.3.1 Best-first search ....................................................................................................... 42
4.3.3.2 The A algorithm ....................................................................................................... 43
4.3.3.3 The A* algorithm ..................................................................................................... 47
4.3.3.4 The monotone A algorithm ...................................................................................... 49
4.3.3.5 The connection among the different variants of the A algorithm ............................ 51
5 2-player games ................................................................................................................................ 52
5.1 State-space representation ....................................................................................................... 53
5.2 Examples ................................................................................................................................. 53
5.2.1 Nim .................................................................................................................................. 53
5.2.2 Tic-tac-toe ....................................................................................................................... 54
5.3 Game tree and strategy ............................................................................................................ 56
5.3.1 Winning strategy ............................................................................................................. 58
5.4 The Minimax algorithm .......................................................................................................... 58
5.5 The Negamax algorithm .......................................................................................................... 61
5.6 The Alpha-beta pruning .......................................................................................................... 63
6 Using artificial intelligence in education ........................................................................................ 66
6.1 The problem ............................................................................................................................ 66
6.1.1 Non-modifiable searchers ............................................................................................... 67
6.1.2 Backtrack searchers ......................................................................................................... 69
6.1.3 Tree Search Methods ....................................................................................................... 70
6.1.4 Depth-First Method ......................................................................................................... 72
6.1.5 2-Player game programs .................................................................................................. 73
6.2 Advantages and disadvantages ................................................................................................ 74
7 Summary ......................................................................................................................................... 75
8 Example programs .......................................................................................................................... 77
8.1 The AbstractState class ........................................................................................................... 77
8.1.1 Source code ..................................................................................................................... 77
8.2 How to create my own operators? .......................................................................................... 78
8.2.1 Source code ..................................................................................................................... 78
8.3 A State class example: HungryCavalryState ........................................................................... 80
8.3.1 Source code ..................................................................................................................... 80
8.4 Another State class example ................................................................................................... 82
8.4.1 The example source code of the 3 monks and 3 cannibals ............................................. 82
8.5 The Vertex class ...................................................................................................................... 84
8.5.1 Source code ..................................................................................................................... 85
8.6 The GraphSearch class ............................................................................................................ 86
8.6.1 Source code ..................................................................................................................... 86
8.7 The backtrack class ................................................................................................................. 87
8.7.1 Source code ..................................................................................................................... 87
8.8 The DepthFirstMethod class ................................................................................................... 88
8.8.1 Source code ..................................................................................................................... 89
8.9 The Main Program .................................................................................................................. 90
8.9.1 Source code ..................................................................................................................... 90
Bibliography ..................................................................................................................................... 92
1 INTRODUCTION
Surely everyone have thought about what artificial intelligence is? In most cases, the answer from a
mathematically educated colleague comes in an instant: It depends on what the definition is? If
artificial intelligence is when the computer beats us in chess, then we are very close to attain
artificial intelligence. If the definition is to drive a land rover through a desert from point A to point
B, then we are again on the right track to execute artificial intelligence. However, if our expectation
is that the computer should understand what we say, then we are far away from it.
This lecture note uses artificial intelligence in the first sense. We will bring out such „clever”
algorithms, that can be used to solve the so called graph searching problems. The problems that can
be rewritten into a graph search – such as chess – can be solved by the computer.
Alas, the computer will not become clever in the ordinary meaning of the word if we implement
these algorithms, at best, it will be able to systematically examine a graph in search of a solution. So
our computer remains as thick as two short planks, but we exploit the no more than two good
qualities that a computer has, which are:
The computer can do algebraic operations (addition, subtraction, etc.) very fast.
It does these correctly.
So we exploit the fact that such problems that are to difficult for a human to see through – like the
solution of the Rubik Cube – are represented in graphs, which are relatively small compared to the
capabilities of a computer, so quickly and correctly applying the steps dictated by the graph search
algorithms will result in a fast-solved Cube and due to the correctness, we can be sure that the
solution is right.
At the same time, we can easily find a problem that's graph representation is so huge, that even the
fastest computers are unable to quickly find a solution in the enormous graph. This is where the
main point of our note comes in: the human creativity required by the artificial intelligence. To
represent a problem in a way that it's graph would keep small. This is the task that should be started
developing in high school. This requires the expansion of the following skills:
Model creation by the abstraction of the reality
System approach
It would be worthwhile to add algorithmic thinking to the list above, which is required to think over
and execute the algorithms published in this note. We will talk about this in a subsequent chapter.
The solution of a problem is the following in the case of applying artificial intelligence:
We model the real problem.
We solve the modelled problem.
With the help of the solution found in the model, we solve the real problem.
All steps are helped by different branches of science. At the first step, the help comes from the
sciences that describe reality: physics, chemistry, etc. The second step uses an abstract idea system,
where mathematics and logic helps to work on the abstract objects. At last, the engineering sciences,
informatics helps to plant the model's solution into reality.
This is all nice, but why can't we solve the existing problem in reality at once? Why do we need
modelling? The answer is simple. Searching can be quite difficult and expensive in reality. If the
well-know 8 Queens Problem should be played with 1-ton iron queens, we would also need a
massive hoisting crane, and the searching would take a few days and a few hundreds of diesel oil till
we find a solution. It is easier and cheaper to search for a solution in an abstract space. That is why
we need modelling.
What guarantees that the solution found in the abstract space will work in reality? So, what
guarantees that a house built this way will not collapse? This is a difficult question. For the answer,
let's see the different steps in detail.
Modelling the existing problem:
We magnify the parts of the problem that are important for the solution and neglect the ones
that are not.
We have to count and measure the important parts.
We need to identify the possible „operators” that can be used to change reality.
Modelling the existing problem is called state space representation in artificial intelligence. We have
a separate chapter on this topic. We are dealing with this question in connection with the „will-the-
house-collapse” - issue. Unfortunately, a house can be ruined at this point, because if we neglect an
important issue, like the depth of the wall, the house may collapse. How does this problem, finding
the important parts in a text, appear in secondary school? Fortunately, it's usually a maths exercise,
which rarely contains unnecessary informations. The writer of the exercise usually takes it the other
way round and we need to find some additional information which is hidden in the text.
It is also important to know that measuring reality is always disturbed with errors. With the tools of
Numeric mathematics, the addition of the the initial errors can be given, so the solution's error
content can also be given.
The third step, the identification of the „operators”, is the most important in the artificial
intelligence's aspect. The operator is a thing, that changes the part of reality that is important for us,
namely, it takes from one well-describable state into another. Regarding artificial intelligence, it's an
operator, when we move in chess, but it may not if we chop down a tree unless the number of the
trees is not an important detail in the solution of the problem.
We will see that our model, also know as state space can be given with
the initial state,
the set of end states,
the possible states and
the operators (including the pre and post condition of the operators).
We need to go through the following steps to solve the modelled problem:
Chose a framework that can solve the problem.
Set the model in the framework.
The framework solves the problem.
Choosing the framework that is able to solve our model means choosing the algorithm that can
solve the modelled problem. This doesn't mean that we have to implement this algorithm. For
example, the Prolog interpreter uses backtrack search. We only need to implement, which is the
second step, the rules that describe the model in Prolog. Unfortunately, this step is influenced by the
fact, that we either took transformational- (that creates a state from another state) or problem
reduction (that creates more states from another state) operators in the state space representation. So
we can take the definition of the operators to be the next step after choosing the framework. The
frameworks may differ from each other in many ways, the possible groupings are:
Algorithms, that surly find the solution in a limited, non-circle graph.
Algorithms, that surly find the solution in a limited graph.
Algorithms, that give an optimal solution according to some point of view.
If we have the adequate framework, our last task is to implement the model in the framework. This
is usually means setting the initial state, the end condition and the operators (with pre- and
postconditions). We only need to push the button, and the framework will solve the problem if it is
able to do it. Now, assume that we have got a solution. First of all, we need to know what do we
mean under 'solution'. Solution is a sequence of steps (operator applications), that leads from the
initial state into an end state. So, if the initial state is that we have enough material to build a house
and the end state is that a house had been built according to the design, then the solution is a
sequence of steps about how to build the house.
There is only one question left: will the house collapse? The answer is definitely 'NO', if we haven't
done any mistake at the previous step, which was creating the model, and will not do at the next
step, which is replanting the abstract model into reality. The warranty for this is the fact that the
algorithms introduced in the notes are correct, namely by logical methods it can be proven that if
they result in a solution, that is a correct solution inside the model. Of course, we can mess up the
implementation of the model (by giving an incorrect end condition, for example), but if we manage
to evade this tumbler, we can trust our solution in the same extent as we can trust in logics.
The last step is to solve the real problem with the solution that we found in the model. We have no
other task than executing the steps of the model's solution in reality. Here, we can face that a step,
that was quite simple in the model (like move the queen to the A1 field) is difficult if not impossible
in reality. If we found that the step is impossible, than our model is incorrect. If we don't trust in the
solution given by the model, then it worth trying it in small. If we haven't messed up neither of the
steps, then the house will stand, which is guaranteed by the correctness of the algorithms and the
fact that logic is based on reality!
2 THE HISTORY OF ARTIFICIAL INTELLIGENCE
Studying the intelligence is one of the most ancient scientific discipline. Philosopher have been
trying to understand for more than 2000 years what mechanism we use to sense, learn, remember,
and think. From the 2000 years old philosophical tradition the theory of reasoning and learning have
developed, along with the view that the mind is created by the functioning of some physical system.
Among others, these philosophical theories made the formal theory of logic, probability, decision-
making, and calculation develop from
mathematics.
The scientific analysis of skills in
connection with intelligence was turned
into real theory and practice with the
appearance of computers in the 1950s.
Many thought that these „electrical
masterminds” have infinite potencies
regarding executing intelligence. „Faster
than Einstein” - became a typical
newspaper article. In the meantime,
modelling intelligent thinking and
behaving with computers proved much
more difficult than many have thought at
the beginning.
The Artificial Intelligence (AI) deals with
the ultimate challenge: How can a (either
biological or electronic) mind sense,
understand, foretell, and manipulate a
world that is much larger and more
complex than itself? And what if we would
like to construct something with such
capabilities?
AI is one of the newest field of science.
Formally it was created in 1956, when its
name was created, although some
researches had already been going on for 5
years. AI's history can be broken down into three major periods.
2.1 EARLY ENTHUSIASM, GREAT EXPECTATIONS (TILL THE END
OF THE 1960S)
In a way, the early years of AI were full of successes. If we consider the primitive computers and
programming tools of that age, and the fact, that even a few years before, computers were only
though to be capable of doing arithmetical tasks, it was astonishing to think that the computer is –
even if far from it – capable of doing clever things.
In this era, the researchers drew up ambitious plans (world champion chess software, universal
translator machine) and the main direction of research was to write up general problem solving
methods. Allen Newell and Herbert Simon created a general problem solving application (General
Figure 1. The early optimism of the 1950s: „The smallest
electronic mind of the world” :)
Program Solver, GPS), which may have been the first software to imitate the protocols of human-
like problem solving.
This was the era when the first theorem provers came into existence. One of these was Herbert
Gelernter's Geometry Theorem Prover, which proved theorems based on explicitly represented
axioms.
Arthur Samuel wrote an application that played Draughts and whose game power level reached the
level of the competitors. Samuel endowed his software with the ability of learning. The application
played as a starter level player, but it became a strong opponent after playing a few days with itself,
eventually becoming a worthy opponent on strong human race. Samuel managed to confute the fact
that a computer is only capable of doing what it was told to do, as his application quickly learnt to
play better than Samuel himself.
In 1958, John McCarthy created the Lisp programming language, which outgrew into the primary
language of AI programming. Lisp is the second oldest programming language still in use today.
2.2 DISILLUSIONMENT AND THE KNOWLEDGE-BASED SYSTEMS (TILL
THE END OF THE 1980S)
The general-purpose softwares of the early period of AI were only able to solve simple tasks
effectively and failed miserably when they should be used in a wider range or on more difficult
tasks. One of the sources of difficulty was that early softwares had very few or no knowledge about
the problems they handled, and achieved successes by simple syntactic manipulations. There is a
typical story in connection with the early computer translations. After the Sputnik's launch in 1957,
the translations of Russian scientific articles were hasted. At the beginning, it was thought that
simple syntactic transformations based on the English and Russian grammar and word substitution
will be enough to define the precise meaning of a sentence. According to the anecdote, when the
famous „The spirit is willing, but the flesh is weak” sentence was re-translated, it gave the
following text: „The vodka is strong, but the meat is rotten.” This clearly showed the experienced
difficulties, and the fact that general knowledge about a topic is necessary to resolve the
ambiguities.
The other difficulty was that many problems that were tried to solve by the AI were untreatable. The
early AI softwares were trying step sequences based on the basic facts about the problem that
should be solved, experimented with different step combinations till they found a solution. The
early softwares were usable because the worlds they handled contained only a few objects. In
computational complexity theory, before defining NP-completeness (Steven Cook, 1971; Richard
Karp, 1972), it was thought that using these softwares for more complex problems is just matter of
faster hardware and more memory. This was confuted in theory by the results in connection with
NP-completeness. In the early era, AI was unable to beat the „combinatorial boom” – combinatorial
explosion and the outcome was the stopping of AI research in many places.
From the end of the 1960s, developing the so-called expert systems were emphasised. These
systems had (rule-based) knowledge base about the field they handled, on which an inference
engine is executing deductive steps. In this period, serious accomplishments were born in the theory
of resolution theorem proving (J. A. Robinson, 1965), mapping out knowledge representation
techniques, and on the field of heuristic search and methods for handling uncertainty. The first
expert systems were born on the field of medical diagnostics. The MYCIN system, for example,
with its 450 rules, reached the effectiveness of human experts, and put up a significantly better
show than novice physicians.
At the beginning of the 1970s, Prolog, the logical programming language were born, which was
built on the computerized realization of a version of the resolution calculus. Prolog is a remarkably
prevalent tool in developing expert systems (on medical, judiciary, and other scopes), but natural
language parsers were implemented in this language, too. Some of the great achievements of this
era is linked to the natural language parsers of which many were used as database-interfaces.
2.3 AI BECOMES INDUSTRY (SINCE 1980)
The first successful expert system, called R1, helped to configure computer systems, and by 1986, it
made a 40 million dollar yearly saving for the developer company, DEC. In 1988, DEC's AI-group
already put on 40 expert systems and was working on even more.
In 1981, the Japanese announced the „fifth generation computer” project – a 10-year plan to build
an intelligent computer system that uses the Prolog language as a machine code. Answering the
Japanese challenge, the USA and the leading countries of Europe also started long-term projects
with similar goals. This period brought the brake-through, when the AI stepped out of the
laboratories and the pragmatic usage of AI has begun. On many fields (medical diagnostics,
chemistry, geology, industrial process control, robotics, etc.) expert systems were used and these
were used through a natural language interface. All in all, by 1988, the yearly income of the AI
industry increased to 2 billion dollars.
Besides expert systems, new and long-forgotten technologies have appeared. A big class of these
techniques includes statistical AI-methods, whose research got a boost in the early years of the
1980's from the (re)discovery of neural networks. The hidden Markov-models, which are used in
speech- and handwriting-recognition, also fall into this category. There had been a mild revolution
on the fields of robotics, machine vision, and learning.
Today, AI-technologies are very versatile: they mostly appear in the industry, but they also gain
ground in everyday services. They are becoming part of our everyday life.
3 PROBLEM REPRESENTATION
3.1 STATE-SPACE REPRESENTATION
The first question is, how to represent a problem that should be solved on computer. After
developing the details of a representation technology, we can create algorithms that work on these
kind of representations. In the followings, we will learn the state-space representation, which is a
quite universal representation technology. Furthermore, many problem solving algorithms are
known in connection with state-space representation, which we will be review deeply in the 3
rd
chapter.
To represent a problem, we need to find a limited number of features and parameters (colour,
weight, size, price, position, etc.) in connection with the problem that we think to be useful during
the solving. For example, if these parameters are described with the values
h
1
, ... , h
n
(colour:
black/white/red; temperature: [-20C˚, 40C˚]; etc.), then we say that the problem's world is in
thestate identified by the vector (
h
1
, ... , h
n
) . If we denote the set which consists of values adopted
by the i. parameter with
H
i
, then the states of the problem's world are elements of the set
H
1
×H
2
×.×H
n
.
As we've determined the possible states of the problem's word this way, we have to give a special
state that specifies the initial values of the parameters in connection with the problem's world. This
is called the initial state.
During the problem-solving, starting from the initial state, we will change the different states of the
problem's world again and again, till we reach an adequate state called the goal state. We can even
define several goal states.
Now we only need to specify which states can be changed and what states will these changes call
forth. The functions that describe the state-changes are called operators. Naturally, an operator can't
be applied to each and every state, so the domain of the operators (as functions) is given with the
help of the so-called preconditions.
Definition 1. A state-space representation is a tuple 〈 A, k ,C ,O〉 , where:
(1) A : is the set of states, A≠∅ ,
(2) k ∈A : is the initial state,
(3) C⊆A : is the set of goal states,
(4) O : is the set of the operators, O≠∅ .
Every o∈O operator is a function o: Dom(o)-A , where
Dom(o)=
¦
a

a∉C ∧ precondition
o
(a)
¦
⊆A
The set C can be defined in two ways:
▫ By enumeration (in an explicit way):
C=
¦
c
1
,., c
m
¦
▫ By formalizing a goal condition (in an implicit way): C=¦a ∣ goal condition(a)¦
The conditions
precondition
o
(a)
and goal condition(a) can be specified as logical formulas.
Each formulas' parameter is a state a , and the precondition of the operator also has the applicable
operator o .
Henceforth, we need to define what we mean the solution of a state-space represented problem – as
that is the thing we want to create an algorithm for. The concept of a problem's solution can be
described through the following definitions:
Definition 2. Let 〈 A, k ,C ,O〉 be a state-space representation, and a , a' ∈A are two states.
a ' is directly accessible from a if there is an operator o∈O where
precondition
o
(a)
holds
and o(a)=a' .
Notation:
a -
o
a '
.
Definition 3. Let 〈 A, k ,C ,O〉 be a state-space representation, and a , a' ∈A are two states.
a ' is accessible from a if there is a
a
1
, a
2
,., a
n
∈A
state sequence where
▫ a
1
=a ,
▫ a
n
=a' ,
▫ ∀ i ∈¦1,2,., n−1¦ :
a
i
-
o
i
a
i+1 (any o
i
∈O operator)
Notation:
a -
o
1
, o
2
,., o
n−1
a'
Definition 4. The problem 〈 A, k ,C ,O〉 is solvable if
k -
o
1
,., o
n
c
for any goal state c∈C . In
this case, the operator sequence
o
1
,., o
n
is referred as a solution to the problem.
Some problems may have more than one solution. In such cases, it can be interesting to compare the
solutions by their costs - and select the less costly (the cheapest) solution. We have the option to
assign a cost to the application of an operator to the state a , and denote it as
cost
o
(a)
(assuming
that o is applicable to a , that is,
precondition
o
(a)
holds), which is a positive integer.
Definition 5. Let
k -
o
1
,., o
n
c
in the case of a 〈 A, k ,C ,O〉 problem for any c∈C . The cost of
the solution
o
1
,., o
n
is:

i=1
n
cost (o
i
, a
i
) .
Namely, the cost of a solution is the cost of all the operator applications in the solution. In the case
of many problems, the cost of operator applications is uniform, that is cost ( o, a)=1 for any
operator o and state a . In this case, the cost of the solution is implicitly the number of applied
operators.
3.2 STATE-SPACE GRAPH
The best tool to demonstrate the state-space representation of a problem is the state-space graph.
Definition 6. Let 〈 A, k ,C ,O〉 be the state-space representation of a problem. The problem's
state-space graph is the graph
1
〈 A, E〉 , where (a , a ' )∈E and (a , a ' ) is labelled with o if
and only if
a -
o
a '
.
Therefore, the vertices of the state-space graph are the states themselves, and we draw an edge
between two vertices if and only if one vertex (as a state) is directly accessible from another vertex
(as a state). We label the edges with the operator that allows the direct accessibility.
It can be easily seen that a solution of a problem is nothing else but a path that leads from a vertex
k (aka the initial vertex) to some vertex c∈C (aka the goal vertex). Precisely, the solution is the
sequence of labels (operators) of the edges that formulate this path.
In Chapter 4, we will get to know a handful of problem solving algorithms. It can be said, in
general, that all of them explore the state-space graph of the given task in different degree and look
for the path that represents the solution in the graph.
1 As usual:
remove mo u n d, p c s
is the set of the graph's vertices,
( a1 ,…, an , p)
is the set of the graph's edges.
3.3 EXAMPLES
In this chapter, we introduce the possible state-space representations of several noteworthy
problems.
3.3.1 3 JUGS
We have 3 jugs of capacities 3, 5, and 8 litres, respectively. There is no scale on the jugs, so it's only
their capacities that we certainly know. Initially, the 8-litre jug is full of water, the other two are
empty:
We can pour water from one jug to another, and the goal is to have exactly 4 litres of water in any of
the jugs. The amount of water in the other two jugs at the end is irrelevant. Here are two of the
possible goal states:
Since there is no scale on the jugs and we don't have any other tools that would help, we can pour
water from jug A to jug B in two different ways:
▫ We pour all the water from jug A to jug B .
▫ We fill up jug B (and it's possible that some water will remain in jug A ).
Give a number to each jug: let the smallest be 1, the middle one 2, and the largest one 3! Generalize
the task to jugs of any capacity: introduce a vector with 3 components (as a constant object out of
the state-space), in which we store the capacities of the jugs:
max=(3,5,8)
Set of states: In the states, we store the amount of water in the jugs. Let the state be an
tuple, in which the i
th
part tells about the jug denoted by i how many litres of water it is
containing.
So, the set of states is defined as follows:
A=
¦
( a
1,
a
2,
a
3
)

0≤a
i
≤max
i
¦
where every a
i
is an integer.
Initial state: at first, jug 3 is full all, the other ones are empty. So, the initial state is:
k=(0,0 , max
3
)
Set of goal states: We have several goal states, so we define the set of goal states with
help of a goal condition:
C=
¦
(a
1
, a
2
, a
3
)∈A

∃i a
i
=4
¦
Set of operators: Our operators realize the pouring from one jug (denoted by i ) to
another one (denoted by j ). We can also specify that the source jug ( i ) and the goal jug ( j )
can't be the same. Our operators are defined as follows:
O=
¦
pour
i , j

i , j ∈¦1,2,3¦ ∧ i ≠j
¦
Precondition of the operators: Let's define when an operator
pour
i , j can be
applied to a state (a
1
, a
2
, a
3
) ! It's practical to specify the following conditions:
▫ Jug i is not empty.
▫ Jug j is not filled.
So, the precondition of the operator
pour
i , j
to the state (a
1
, a
2
, a
3
) is:
a
i
≠0 ∧ a
j
≠max
j
Function of appl ying: Define what state
(a'
1
, a'
2
, a'
3
)
does the operator
pour
i , j

create from the state (a
1
, a
2
, a
3
) ! The question is how many litres of water can we pour from
jug i to jug j . Since at most
max
j
−a
j
litres of water can be poured to jug j , we can calculate the exact amount to be poured by
calculating
min(a
i
, max
j
−a
j
)
Denote this amount with T . Consequently:
pour
i , j
(a
1
, a
2
, a
3
)=(a '
1
, a'
2
, a'
3
)
, where
a '
m
=
¦
a
i
−T , if m=i
a
j
+T , if m=j
a
m
, otherwise
where m∈¦1,2,3¦
STATE-SPACE GRAPH
The state-space graph of the aforementioned state-space representation can be seen in Figure 2.
In the graph, the red lines depict unidirectional edges while the green ones are bidirectional edges.
Naturally, bidirectional edges should be represented as two unidirectional edges, but due to lack of
space, let us use bidirectional edges. It can be seen that the labels of the bidirectional edges are
given in the form of
pour
i , j1− j2
, which is different from the form of
pour
i , j
as it was given in the
state-space representation. The reason for this is that one
pour
i , j1− j2
label encodes two operators at
the same time: the operators
pour
i , j1
and
pour
i , j2
.
The green vertices represent the goal states. The bold edges represent one of the solutions, which is
the following operator sequence:

pour
3,2
, pour
2,1
, pour
1,3
, pour
2,1
, pour
3,2
, pour
2,1
Notice that the problem has several solutions. It can also be noticed that the state-space graph
contains cycles , that makes it even more difficult to find a solution.
Figure 2. The state-space graph of the 3 Jugs problem.
3.3.2 TOWERS OF HANOI
There are 3 discs with different diameters. We can slide these discs onto 3 perpendicular rods. It's
important that if there is a disc under another one then it must be bigger in diameter. We denote the
rods with „P”, „Q”, and „R”, respectively. The discs are denoted by „1”, „2”, and „3”, respectively,
in ascending order of diameter. The initial state of discs can be seen in the figure below:
We can slide a disc onto another rod if the disc
(5) is on the top of its current rod, and
(6) the discs on the goal rod will be in ascending order by size after the replacing.
Our goal is to move all the discs to rod R.
We create the state-space representation of the problem, as follows:
Set of states: In the states, we store the information about the currently positions (i.e.,
rods) of the discs. So, a state is a vector
(a
1
, a
2
, a
3
)
where
a
i
is the position of disc i (i.e.,
either P, Q, or R). Namely:
A=
¦
( a
1
, a
2
, a
3
)

a
i
∈¦P , Q, R¦
¦
Initial state: Initially, all the discs are on rod P, i. e.:
k=( P , P , P)
Set of goal states: The goal is to move all the three discs to rod R. So, in this problem,
we have only one goal state, namely:
C=¦( R, R , R)¦
Set of operators: Each operator includes two pieces of information:
▫ which disc to move
▫ to which rod?
Namely:
O=
¦
move
which, where

which∈¦1,2,3¦ , where∈¦P ,Q , R¦
¦
Precondition of the operators: Take an operator
move
which ,where ! Let's examine
when we can apply it to a state
(a
1
, a
2
, a
3
)
! We need to formalize the following two
conditions:
(1) The disc which is on the top of the rod
a
which
.
(2) The disc which is getting moved to the top of the rod where .
What we need to formalize as a logical formula is that each disc that is smaller than disc
which (if such one does exist) is not on either rod
a
which
or rod where .
It's worth to extend the aforementioned condition with another one, namely that we don't want
to move a disc to the same rod from which we are removing the disc. This condition is not
obligatory, but can speed up the search (it will eliminate trivial cycles in the state-space graph).
Thus, the precondition of the operators is:
a
which
≠where∧∀ i
(
i which ⇒ a
i
≠a
which
∧ a
i
≠where
)
Function of appl ying: Take any operator
move
which ,where ! If the precondition of the
operator holds to a state
(a
1
, a
2
, a
3
)
, then we can apply it to this state. We have to formalize
that how the resulting state
(a'
1
, a'
2
, a'
3
)
will look like.
We have to formalize that the disc which will be moved to the rod where , while the other
discs will stay where they currently are. Thus:
move
which ,where
(a
1
, a
2
, a
3
)=(a '
1
, a'
2
, a'
3
)
, where
a '
i
=
¦
where , if i =which
a
i
, otherwise
where i ∈¦1,2,3¦
Important note: we have to define all of the components of the new state, not just the one that
changes!
STATE-SPACE GRAPH
The state-space graph of the aforementioned state-space representation can be seen in Figure 3.
Naturally, all of the edges in the graph are bidirectional, and their labels can be interpreted as in the
previous chapter: a label
move
i , j1− j2
refers to both the operators
move
i , j1
and
move
i , j2
.
As it can be clearly seen in the figure, the optimal (shortest) solution of the problem is given by the
Figure 3. The state-space graph of the Towers of Hanoi problem.
rightmost side of the large triangle, namely, the optimal solution consists of 7 steps (operators).
3.3.3 8 QUEENS
Place 8 queens on a chessboard in a way that no two of them attack each other. One possible
solution:
Generalize the task to a N×N ( N¯1) chessboard, on which we need to place N queens. N is
given as a constant out of the state-space.
The basic idea of the state-space representation is the following: since we will place exactly one
queen to each row of the board, we can solve the task by placing the queens on the board row by
row. So, we place one queen to the 1
st
row, then another one to the 2
nd
row in a way that they can't
attack each other. In this way, in step i
th
we place a queen to row i while checking that it does not
attack any of the previously placed i−1 queens.
Set of states: In the states we store the positions of the placed queens within a row! Let's
have a N -component vector in a state, in which component i tells us to which column in row
i a queen has been previously placed. If we haven't placed a queen in the given row, then the
vector should contain 0 there.
In the state we also store the row in which the next queen will be placed.
So:
A=
¦
( a
1
, a
2
,., a
N
, s)

0<a
i
<N ,1<s<N+1
¦
As one of the possible value of s , N+1 is a non-existent row index, which is only permitted
for testing the terminating conditions.
Initial state: Initially, the board is empty. Thus, the initial state is:
k=(0,0 ,.,0,1)
Set of goal states:
We have several goal states. If the value of s is a non-existent row index, then we have found
a solution. So, the set of goal states is:
C=
¦
(a
1
, ., a
N
, N +1)∈A
¦
Set of operators:
Our operators will describe the placing of a queen to row s . The operators are expecting only
one input data: the column index where we want to place the queen in row s . The set of our
operators is:
O=
¦
place
i

1<i <8
¦
Precondition of the operators: Formalize the precondition of applying an operator
place
i
to a state (a
1
,., a
8
, s) ! It can be applied if the queen we are about to place is
▫ not in the same row as any queens we have placed before. So, we need to examine if the value
of i was in the state before the s
th
component. i. e.,
for all ms: a
m
≠i
▫ not attacking any previously placed queens diagonally. Diagonal attacks are the easiest to
examine if we take the absolute value of the difference of the row indices of two queens, and
then compare it to the absolute value of the difference of the column indices of the two
queens. If these values are equal then the two queens are attacking each other. The row index
of the queen we are about to place is s , while the column-index is i . So:
for all ms : ∣s−m∣≠∣i −a
m

Thus, the precondition of the operator
place
i
to the state (a
1
,., a
8
, s) is:
∀ m
(
ms ⇒ a
m
≠i ∧ ∣s−m∣≠∣i−a
m

)
,
where 1<m<8
Function of appl ying: Let's specify the state
(a'
1
,., a'
8
, s' )
which the operator
place
i
will create from a state (a
1
,., a
8
, s) ! In the new state, as compared to the original
state, we only need to make the following modifications:
▫ write i to the s
th
component of the state, and
▫ increment the value of s by one.
Thus:

place
i
(a
1
,., a
8
, s)=(a '
1
,., a '
8
, s ' )
, where
a '
m
=
¦
i , if m=s
a
m
, otherwise
, where m∈¦1,2,3¦
s' = s+1
STATE-SPACE GRAPH
The state-space graph of the aforementioned state-space representation for the case N=4 case can
be seen in Figure 4. In this case, the problem has 2 solutions.
Notice that every solution of the problem is N long for sure. It is also important to note that there is
no cycle in the state-space graph, that is, by carefully choosing the elements of the state-space
representation, we have managed to exclude cycles from the graph, which will be quite
advantageous when we are searching solutions.
Figure 4. The state-space graph of the 4 Queens problem.
4 PROBLEM-SOLVING METHODS
Problem-solving methods are assembled from the following components:
Database: stored part of the state-space graph.
As the state-space graph may contain circles (and loops), in the database we store the graph in
an evened tree form (see below).
Operations: tools to modify the database.
We usually differentiate two kinds of operations:
▫ operations originated from operators
▫ technical operations
Control ler: it controls the search in the following steps:
(1) initializing the database,
(2) selecting the part of the database that should be modified,
(3) selecting and executing an operation,
(4) examining the terminating conditions:
• positive terminating: we have found a solution,
• negative terminating: we appoint that there is no solution.
The controller usually executes the steps between (1) and (4) iteratively.
UNFOLDING THE STATE-SPACE GRAPH INTO A TREE
Let's see the graph on Figure 5. The graph contains cycles, one such trivial cycle is the edge from s
to s or the path s , c , b , s and the path c , d , b , s , c . We can eliminate the cycles from the graph
by duplicating the appropriate vertices. It can be seen on Figure 6, for example, we eliminated the
edge from s to s by inserting s to everywhere as the child of s . The s , c , b , s cycle appears on
the figure as the rightmost branch. Of course, this method may result in an infinite tree, so I only
give a finite part of it on the figure.
After unfolding, we need to filter the duplications on the tree branches if we want the solution
seeking to terminate after a limited number of steps. That's we will be using different cycle
detection techniques (see below) in the controller.
Although, they do not endanger the finiteness of the search, but the multiple paths in the state-space
graph do increase the number of vertices stored in the database. On Figure 5, for example, the c , d
and the c , b , d paths are multiple, as we can use either of them to get from c to d . The c , d and
c , b , a , d paths are also multiple in a less trivial way. On Figure 6, we can clearly see what
Figure 6. The unfolded tree version.
Figure 5. A graph that contains cycles and
multiple paths.
multiple paths become in the tree we got: the vertices get duplicated, although, not on the same
branches (like in the case of cycles), but on different branches. For example, the vertex d appears
three times on the figure, which is due to the previously mentioned two multiple paths. Note that the
existence of multiple paths not only results in the duplication of one or two vertices, but the
duplication of some subtrees: the subtree starting at b and ending at the vertices
(a ' 1 ,…, a' n , p ' )
appears
twice on the figure.
As I already mentioned, the loops do not endanger the finiteness of the search. But it is worth to
use some kind of cycle detection technique in the controller if it holds out a promise of sparing
many vertices, as we reduce the size of the database on a large scale and we spare the drive.
Moreover, this last thing entails the reduction of the runtime.
THE FEATURES OF PROBLEM-SOLVING METHODS
In the following chapter we will get to know different problem-solving methods. These will
differentiate from each other in the composition of their databases, in their operations and the
functioning of their controllers. These differences will result in problem-solving methods with
different features and we will examine the following of these features in the case of every such
method:
Completeness: Will the problem-solving method stop after a finite number of steps on
every state-space representation, will it's solution be correct or does a solution even exist for
the problem? More clearly:
▫ If there is a solution, what state-space graph do we need for a solution?
▫ If there is no solution, what state-space graph will the problem-solving method need to
recognize this?
We will mostly differentiate the state-space graphs by their finiteness. A graph is considered
finite if it does not contain a circle.
Optimality: If a problem has more than one solution, does the problem-solving method
produce the solution with the lowest cost?
THE CLASSIFICATION OF PROBLEM-SOLVING METHODS
The problem-solving methods are classified by the following aspects:
Is the operation retractable?
(1) Non-modifiable problem-sol ving methods: The effects of the operations
cannot be undone. This means that during a search, we may get into a dead end from which
we can't go back to a previous state. The advantage of such searchers is the simple and
small database.
(2) Modifiable problem/sol ving methods: The effects of the operations can be
undone. This means that we can't get into a dead end during the search. The cost of this is
the more complex database.
How does the controller choose from the database?
(1) Systematic problem-sol ving methods: Randomly or by some general
guideline (e.g. up to down, left to right). Universal problem-solving methods, but due to
their blind, systematic search strategy they are ineffective and result in a huge database.
(2) Heuristic problem-sol ving methods: By using some guessing, which is done
on the basis of knowledge about the topic by the controller. The point of heuristics is to
reduce the size of the database so the problem-solving method will become effective. On the
other hand, the quality of heuristics is based on the actual problem, there is no such thing as
„universal heuristics”.
4.1 NON-MODIFIABLE PROBLEM-SOLVING METHODS
The significance of non-modifiable problem-solving methods is smaller, due to their features they
can be rarely used, only in the case of certain problems. Their vital advantage is their simplicity.
They are only used in problems where the task is not to find a solution (as a sequence of operators),
but to decide if there is a solution for the problem and if there is one, than to create a (any kind of)
goal state.
The general layout of non-modifiable sproblem-solving methods:
Database: consists of only one state (the current state).
Operations: operators that were given in the state-space representation.
Control ler: The controller is trying to execute an operator on the initial state and will
overwrite the initial state in the database with the resulting state. It will try to execute an
operator on the new state and it will again overwrite this state. Executing this cycle continues
till the current state happens to be a goal state. In detail:
(1) Initiating: Place the initial state in the database.
(2) Iteration:
(a) Testing: If the current state (marked with ) is a goal state then the search stops. A solution
exists.
(b) Is there an operator that can be executed on ?
• If there isn't, then the search stops. We haven't found a solution.
• If there is, then mark it with . Let o(a) ( be the current state.
The features of non-modifiable problem-solving methods:
Completeness:
▫ Even if there is a solution, finding it is not guaranteed.
▫ If there is no solution, it will recognize it in the case of a finite state-space graph.
Optimality: generating the optimal goal state (the goal state that can be reached by the
optimal solution) is not guaranteed.
The certain non-modifiable problem-solving methods differ in the way they choose their operator
o for the state a . Wemention two solutions:
(1) Trial and Error method: o is chosen randomly.
Figure 7. The flowchart of a non-modifiable problem-solving method.
(2) Hill climbing method: We choose the operator that we guess to lead us the closest to
any of the goal states.
The magnitude of non-modifiable problem-solving methods is that they can be restarted. If the
algorithm reaches a dead end, that is, there is no operator we can use for the current state, then we
simply restart the algorithm (RESTART). In the same time, we replenish the task to exclude this
dead end (which can be most easily done by replenishing the precondition of the operator leading to
the dead end). We set the number of restarts in advance. It's foreseeable that by increasing the
number of restarts, the chance for the algorithm to find a solution also increases – provided that
there is a solution. If the number of restarts approaches infinity, than the probability of finding a
solution approaches 1.
The non-modifiable problem-solving algorithms that use restarts are called restart algorithms.
The non-modifiable problem-solving methods are often pictured with a ball thrown into a terrain
with mountains and valleys, where the ball is always rolling down, but bouncing a bit before
stopping at the local minimum. According to this, our heuristics chooses the operator that brings to
a smaller state in some aspect (rolling down), but if there is no such option, than it will randomly
select an operator (bouncing) till it is discovered that the ball will roll back to the same place. This
will be the local minimum.
In this example, restart means that after finding a local minimum, we throw the ball back again to a
random place.
In the restart method, we accept the smallest local minimum we have found as the approached
global minimum. This approach will be more accurate if the number of restarts is greater.
The non-modifiable algorithms with restart have great significance in solving the SAT problem. The
so-called „random walk” SAT solving algorithms use these methods.
4.1.1 THE TRIAL AND ERROR METHOD
As it was mentioned above, in the case of the trial and error method, we apply a random operator on
the current vertex.
Completeness:
▫ Even if there is a solution, finding it is not guaranteed.
▫ If there is no solution, it will recognize it in the case of a finite state-space graph.
Optimality: generating the optimal goal state is not guaranteed.
The (only) advantage of the random selection is: the infinite loop is nearly impossible.
IDEA:
• If we get into a dead end, restart.
• In order to exclude getting into that dead end, note that vertex (augment the database).
4.1.2 THE TRIAL AND ERROR METHOD WITH RESTART
Database: the current vertex, the noted dead ends, the number of restarts and the number of
maximum restarts.
Control ler:
(1) Initiating: The initial vertex is the current vertex, the list of noted dead ends is empty, the
number of restarts is 0.
(2) Iteration: Execute a randomly selected applicable operator on the current vertex. Examine
the new state we got whether it is in the list of known dead ends. If yes, then jump back to
the beginning of the iteration. If no, then let the vertex we got be the current vertex.
(a) Testing: If the current vertex is a terminal vertex, then the solution can be backtracked
from the data written on the screen.
(b) If there is no applicable operator for the current vertex, so the current vertex is a dead end:
• If we haven't reached the number of maximum restarts, the we put the found dead end
into the database, increase the number of restarts by one, let the initial vertex be the
current vertex, and jump to the beginning of the iteration.
• If the number of maximum restarts have been reached, then write that we have found no
solution.
The features of the algorithm:
Completeness:
▫ Even if there is a solution, finding it is not guaranteed.
▫ The greater the number of maximum restarts is, the better the chances are to find the solution.
If the number of restarts approaches infinity, then the chance of finding a solution approaches
1.
▫ If there is no solution, it will recognize it.
Optimality: generating the optimal goal state is not guaranteed.
The trial and error algorithm has theoretical significance. The method with restart is called „random
walk”. The satisfiability of conjunctive normal forms can be most practically examined with this
algorithm.
4.1.3 THE HILL CLIMBING METHOD
The hill climbing method is a heuristic problem-solving method. Because the distance between a
state and goal state is guessed through a so-called heuristics. The heuristics is nothing else
but a function on the set of states ( A ) which tells what approximately the path cost is between a
state and the goal state. So:
Definition 7. The heuristics given for the 〈 A, k ,C ,O〉 state-space representation is a
h: A-ℕ
function, that
h(c)=0
∀ c∈C .
The hill climbing method uses the applicable operator o for the state a where h( o(a)) is
minimal.
Let's see how the hill climbing method works in the case of the Towers of Hanoi! First, give a
possible heuristics for this problem! For example, let the heuristics be the sum of the distance of the
discs from rod R . So:
h(a
1
, a
2
, a
3
)=

i=1
3
( R−a
i
)
where R−P=2 , R−Q=1 , and R−R=0 . Note that for the goal state ( R, R , R) ,
h( R, R , R)=0 holds.
Initially, the initial state ( P , P , P) is in the database. We can apply either the
operator
move
1, Q
or
move
1, R
. The first one will result in the state (Q, P , P) with
heuristics 5, and the later one will result in ( R, P , P) with 4. So ( R, P , P) will
be the current state. Similarly, we will insert ( R, Q, P) into the database in the
next step.
Next, we have to choose between two states: we insert either ( R, P , P) or (Q, Q , P) into the
database. The peculiarity of this situation is that the two states have equal heuristics, and the hill
climbing method doesn't say a thing about how to choose between states having the same heuristics.
So in this case, we choose randomly between the two states. Note that, if we chose ( R, P , P) , then
we would get back to the previous state, from where we again get to ( R, Q, P) , where we again
step to ( R, P , P) , and so on till the end of times. If we choose (Q, Q , P) now, then the search can
go on a hopefully not infinite branch.
Going on this way, we meet a similar situation in the state ( R, Q , R) , as we can
step to the states (Q, Q , R) and ( R, P , R) with equal heuristics. We would again
run into infinite execution with the first one.
We have to say that we need to be quite lucky with this heuristics for hill climbing
method even to stop. Maybe, a more sophisticated heuristics might ensure this, but
there is no warranty for the existence of such heuristics. All in all, we have to see
that without storing the past of the search, it's nearly impossible to complete the
task and evade the dead ends.
Note that the Towers of Hanoi is a typical problem for which applying a non-
modifiable problem-solving method is pointless. The (only one) goal state is
known. In this problem, the goal is to create one given solution and for this a non-modifiable
method is inconvenient by its nature.
4.1.4 HILL CLIMBING METHOD WITH RESTART
The hill climbing method with restart is the same as the hill climbing method with the addition that
we allow a set number of restarts. We restart the hill climbing method if it gets into a dead end. If it
reaches the maximum number of restarts and gets into a dead end, then the algorithm stops because
it haven't found a solution.
It is important for the algorithm to learn from any dead end, so it can't run into the same dead end
twice. Without this, the heuristics would lead the hill climbing method into the same dead end after
a restart, except if the heuristics has a random part. The learning can happen in many ways. The
easiest method is to change the state-space representation in a way that we delete the current state
from the set of states if we run into a dead end. Another solution is to expand the database with the
list of forbidden states.
It is worth to use this method if
1. either it learns, that is, it notes the explored dead ends,
2. or the heuristics is not deterministic.
4.2 BACKTRACK SEARCH
One kind of the modifiable problem-solving methods is the backtrack search, which has several
variations. The basic idea of backtrack search is to not only store the current vertex in the database,
but all the vertices that we used to get to the current vertex. This means that we will store a larger
part of the state-space graph in the database: the path from the initial vertex to the current vertex.
The great advantage of backtrack search is that the search can't run into a dead end. If there is no
further step forward in the graph from the current vertex, then we step back to the parent vertex of
the current one and try to another direction from there. This special step – called the back-step –
gave the name of this method.
It is logical that in the database, besides the stored vertex's state, we also need to store the directions
we tried to step to. Namely, in every vertex we have to register those operators that we haven't tried
to apply for the state stored in the vertex. Whenever we've applied an operator to the state, we delete
it from the registration stored in the vertex.
4.2.1 BASIC BACKTRACK
Database: the path from the initial vertex to the current vertex, in the state-space graph.
Operations:
▫ Operators: they are given in the state-space representation.
▫ Back-step: a technical operation which means the deleting of the lowest vertex of the path
stored in the database.
Control ler: Initializes the database, executes an operation on the current vertex, tests the
goal condition, and decides if it stops the search or re-examines the current vertex. The
controller's action in detail:
(1) Initialization: Places the initial vertex as the only vertex in the database.
Initial vertex = initial state + all the operators are registered.
(2) Iteration:
(a) If the database is empty, the search stops. We haven't found a solution.
(b) We select the vertex that is at the bottom of the path (the vertex that was inserted at latest
into the database) stored in the database; we will call this the current vertex. Let us denote
the state stored in the current vertex with a !
(c) Testing: If a is a goal state then the search stops. The solution we've found is the
database itself (as a path).
(d) Examines if there is an operator that we haven't tried to apply to a . Namely, is there any
more operators registered in the current vertex?
• If there is, denote it with o ! Delete o from the current vertex. Test o 's precondition
on a . If it holds then apply o to a and insert the resulted state o(a) at the bottom of
the path stored in the database. In the new vertex, besides o(a) , register all the
operators.
• If there isn't, then the controller steps back.
The backtrack search we have got has the following features:
Completeness:
▫ If there is a solution, it will find it in a finite state-space graph.
▫ If there is no solution, it will recognize it in the case of a finite state-space graph.
Figure 8. The flowchart of the basic backtrack method.
Optimality: generating the optimal goal state is not guaranteed.
IMPLEMENTATION QUESTIONS
What data structure do we use for the database?
Stack.
The operations of the method can be suited as the following stack operations:
▫ applying an operator ⇐ PUSH
▫ back-step ⇐ POP
In what form do we register the operators i n the vertices?
(1) We store a list of operators in each vertex.
Idea: when inserting a new state, don't store all the operators in the new vertex's list of
operators, only those that can be applied to the given state. We can save some storage this
way, but it can happen that we needlessly test some operators' precondition on some states
(as we may find the solution before their turn).
(2) We store the operators outside the vertices in a constant array (or list). In the vertices, we
only store the operator indices, namely the position (index) of the operator in the above
mentioned array. The advantage of this method is that we store the operators themselves ain
one place (there is no redundancy).
(3) We can further develop the previous solution by that we apply the operators on every state
in the order of their position they occur in the array of operators. With this, we win the
following: it's completely enough to store one operator index in the vertices (instead of the
aforementioned list of operators). This operator index will refer to the operator that we will
apply next to the state stored in the vertex. In this way, we know that the operators on the
left of the operator index in the array of operators have already been applied to the state,
while the ones on the right haven't been applied yet.
In this stack, 3 vertices of the 3-mugs-
problem can be seen. We have tried to apply
the operators
pour
3,1
and
pour
3,2
to the
initial vertex (at the bottom). We have got the
2
nd
vertex by applying
pour
3,2
, and we only
have the operators
pour
1,2
and
pour
3,1
left.
By applying
pour
2,1
, we have got the 3
rd
(uppermost, current) vertex, to which we
haven't tried to apply any operator.
In the case of the 3-mugs-problem, the
(constant) array of the operators
consists of 6 elements. In the vertices,
we store the indices (or references) of
the not-yet-used operators.
The precondition of the back-step can be very easily defined: if the operator index stored in
the current vertex is greater than the size of the operators' array, then we step back.
EXAMPLE:
In case of the Towers of Hanoi problem, the basic backtrack search will get into an infinite loop, as
sooner or later the search will stuck in one of the cycles of the state-space graph. The number of
operator executions depends on the order of the operators in the operators' array.
On Figure 9, we show a few steps of the search. In the upper left part of the figure, the operators'
array can be seen. We represent the stack used by the algorithm step by step, and we also show the
travelled path in the state-space graph (see Figure 3).
As it can be seen, we will step back and forth between the states ( R, P , P) and the (Q, P , P)
while filling up the stack. This happens because we have assigned a kind of priority to the operators,
and the algorithm is strictly using the operator with the highest priority.
We haven't applied any operator to the current vertex,
so let its operator index be 1, noting that next time we
will try to apply the operator with index 1.
To the 2
nd
vertex, we tried to apply the operators in
order, where
pour
2,1
was the last one (the 3
rd
operator). Next time we will try the 4
th
one.
We have tried to apply all the operators to the initial
vertex, so its operator index refers to a non-existent
operator.
4.2.2 BACKTRACK WITH DEPTH LIMIT
One of the opportunities for improving the basic backtrack method is the expansion of the
algorithm's completeness. Namely, we try to expand the number of state-space graphs that the
algorithm can handle.
The basic backtrack method only guarantees stopping after a limited amount of steps in case of a
finite state-space graph. The cycles in the graph endanger the finite execution, so we have to
eliminate them in some way. We get to know two solutions for this: we will have the backtrack
method combined with cycle detection in the next chapter, and a more simple solution in this
chapter, that does not eliminate the cycles entirely but allows to walk along them only a
limited number of times.
We achieve this with a simple solution: maximizing the size of the database. In a state-space graph it
means that we traverse it only within a previously given (finite) depth. In implementation, it means
that we specify the maximum size of the stack ain advance. If the database gets „full” in this sense
Figure 9. The basic backtrack in the case of the Towers of Hanoi.
during the search, then the algorithm steps back.
So let us specify an integer limit >0 . We expand the back-step precondition in the algorithm: if the
size of the database has reached the limit , then we take a back-step.
The resulting backtrack method's features are the following:
Completeness:
▫ If there is a solution and the value of the limit is not smaller than the length of the optimal
solution, then the algorithm finds a solution in any state-space graph.
But if we chose the limit too small, then the algorithm doesn't find any solution even if there
is one for the problem. In this sense, backtrack with depth limit does not guarantee a solution.
▫ If there is no solution, then it will recognize this fact in the case of any state-space graph.
Optimality: generating the optimal solution is not guaranteed.
EXAMPLE
On figure 11, we have set a depth limit, which is 7. The depth limit is indicated by the red line at the
top of the stack on the figure. Let us continue the search from the point where we finished it on
Figure 11, namely the constantly duplicating of the states ( R, P , P) and (Q, P , P) . At the 7
th
step
of the search, the stack's size reaches the depth limit, a back-step happens, which means deleting the
vertex on the top of the stack and applying the next applicable operator to the vertex below it. As it
is clearly visible, the search gets out of the infinite execution, but it can also be seen that it will take
tons of back-steps to head forth the goal vertex.
Note that if we set the depth limit lower than 7, then the algorithm would not find a solution!
Figure 10. The flowchart of the backtrack method with depth limit.
4.2.3 BACKTRACK WITH CYCLE DETECTION
For ensuring that the search is finite, another method is the complete elimination of the cycles in
the state-space graphe. It can be achieved by introducing an additional test: a vertex can only be
inserted as a new one into the database if it hasn't been the part of it yet. That is, any kind of
duplication is to be eliminated in the database.
Figure 11. Backtrack with depth limit in the case of the Towers of Hanoi.
The resulting problem-solving method has the best features regarding completeness:
Completeness:
▫ If there is a solution, the algorithmfinds it in any state-space graph.
▫ If there is no solution, the algorithm realizes this fact in the case of any state-space graph.
Optimality: finding the optimal solution is not guaranteed.
The price of all these great features is a highly expensive additional test. It's important to only use
this cycle detection test in our algorithm if we are sure that there is a cycle in the state-space graph.
It is quite an expensive amusement to scan through the database any time a new vertex is getting
inserted!
EXAMPLE
In Figure 13, we can follow a few steps of the backtrack algorithm with cycle detection, starting
from the last but one configuration in Figure 9. Among the move attempts from the state (Q, P , P)
, there are operators
through
1, R
and
through
1, P
, but the states ( R, P , P) and ( P , P , P) created
by them are already in the stack, so we don't put them in again. This is how we reach the operator
through
2, R
, which creates the state (Q, R , P) , that is not part of the stack yet.
This is the spirit the search is going on, eliminating the duplications in the stack completely. It's
worthwhile to note that although the algorithm cleverly eliminates cycles, it reaches the goal vertex
in a quite dumb way, so the solution it finds will be far from optimal.
Figure 12. The flowchart of the backtrack algorithm with cycle detection.
4.2.4 THE BRANCH AND BOUND ALGORITHM
We can try to improve the backtrack algorithm shown in the last chapter for the following reason: to
guarantee the finding of an optimal solution! An idea for such an improvement arises quite
naturally: the backtrack algorithm should perform minimum selection in the universe of possible
solutions.
So, the algorithm will not terminate when it finds a solution, but will make a copy of the database
(the vector called solution will be used for this purpose), then steps back and continues the search.
It follows that the search will only end if the database gets empty. As we don't want to traverse the
whole state-space graph for this, we are going to use a version of the backtrack algorithm with
Figure 13. Backtrack with circle-checking in the case of the Towers of Hanoi.
depth limit in which the value of the limit dynamically changes. Whenever finding a solution (and
storing it in the vector solution ), the limit 's new value will be the length of the currently found
solution! So, we won't traverse the state-space graph more deeply than the length of the last
solution.
It's obvious that the resulting backtrack algorithm – which is called the Branch and Bound
algorithm – finds an optimal solution (if the given problem has a solution).
At the beginning of the search, the variables solution and limit must be initialized. The solution
is an empty vector at the beginning, and the limit is such a great number that is by all means
greater than the length of the optimal solution. The programmers in most cases set the value of
limit to the largest representable integer value. The Branch and Bound algorithm is usually
combined with cycle detection.
The features of the Branch and Bound algorithm:
Completeness:
▫ If there is a solution, the algorithm finds it in any state-space graph.
▫ If there is no solution, the algorithm realizes this fact in the case of any state-space graph.
Optimality: finding the optimal solution is guaranteed.
4.3 TREE SEARCH METHODS
Another large class of the modifiable problem-solving algorithms is the class of the tree search
methods. The basic difference compared to the backtrack algorithms is that we not only store one
branch of the state-space graph in the database, but a more complex part of it, in the form of a tree.
As a matter of fact, the search is done simultaneously on several branches at the same time, so we
probably get a solution sooner, which is maybe an optimal solution, too. An obvious disadvantage
of this method is its higher space complexity.
In the next chapter, we give the description of a general tree search problem-solving method That
Figure 14. The flowchart of the Branch and Bound algorithm.
method is not a concrete algorithm, but includes the common components of the later described
other algorithms – the Breadth-first method, the Depth-first method, the Uniform-cost method, the
Best-first method, and the A algorithm.
4.3.1 GENERAL TREE SEARCH
Database: It is a part of the state-space graph unfolded into a tree. The vertices of the tree
stored in the database are divided into two classes:
▫ Closed vertices: previously expanded vertices (c.f. operations).
▫ Open vertices: not yet expanded vertices.
Operation: expansion.
Only open vertices can be expanded. Expanding a vertex means the following: all the
applicable operators are applied to the vertex. Practically speaking, all the children vertices (in
the state-space graph) of the given vertex are created.
Control ler: Initializes the database, expands the chosen open vertex, tests the goal
condition, and, according to this, decides either top with the search or to expand another vertex.
In details:
(1) Initialization: Inserts the initial vertex as the only one into the database as an open
vertex.
(2) Iteration:
(a) If there is no open vertex in the database, the search stops. It hasn't found a solution.
(b) It selects one of the open vertices, which is going to be the current vertex. Let us denote
this vertex v !
(c) If v is a goal vertex, the search stops. The found solution is: the path from the initial
vertex to v in the database.
(d) Expands v , inserts the created new states as children of v and open vertices into the
database, then reclassifies v as a closed vertex.
There are only three not completely fixed elements in the aforementioned general tree search
method:
▫ at (2)(a): If there are several open vertices in the database, which one should be chosen for
expansion?
Figure 15. The flowchart of tree search methods.
▫ at (2)(c): The goal condition is tested on the current vertex. Thus, after a vertex has been inserted
into the database, we have to wait with the testing of the goal condition until the given vertex
gets selected for expansion. In case of some problem-solving methods, the testing can be
performed sooner in order to haste the search, which means that we test the goal condition
immediately (before inserting into the database) for the new states that were created in (2)(d).
▫ at (2)(d): Should we use any kind of cycle detection technique, and if we do, which one?
Namely, if any state that comes into being due to expansion occurs in the database, shall we
insert it again?
Two sorts of cycle detection techniques can be used:
▪ We scan the whole database looking for the created state. In this case, we are not only checking
for cycles but also for multiple paths, since there won't be two identical states in the database.
Fully excluding The entire elimination of duplications obviously speeds up the search, but
scanning the whole database all the time is quite a costly procedure.
▪ Only the branch that leads to the current vertex is scanned. In this case, we only check for cycle
and don't exclude multiple paths from the database. This is a less costly procedure than the
previous one, but the database may contain duplications.
Detecting cycles or multiple paths in our algorithm depends on the nature of the solvable problem.
If the problem's state-space graph doesn't contain any cycles, then, naturally, cycle detection is not
needed. Otherwise, a cycle detection must be included by all means. If the graph contains only a
few multiple paths, then it's sufficient to use the less costly procedure, but if it contains many, and
hence, duplications substantially slow the search, then it's expedient to choose the full scan (since
this cycle detecting procedure also checks for multiple paths).
It is matter of by arrangement which open vertex is selected for expansion at (2)(a). This is the point
where the concrete (not general) tree search methods differ from each other. In the next chapters, we
introduce the most important such problem-solving methods, and we examine which one is worth to
use for what kind of problems.
IMPLEMENTATION QUESTIONS
What data structure should be used to realize the vertices of the
database?
Every vertex should store pointers pointing to the children of the vertex. This requires the use
of a vertex list in every vertex. It's more economic to store a pointer pointing to the vertex's
parent, since a vertex can have several children, but only one parent (except for the root vertex,
which does not have any parent).
How should the open and closed vertices be registered?
One option is to store the information about being open or closed in the vertex itself. This
solution implies that we always have to scan for the open vertex we want to expand in the tree.
On the one hand, this is a quite costly method, on the other hand, if we stored, according to the
previous point, the parent pointers, then traversing the tree top-down is impossible.
So, it would be practical to store the vertices in one more list. We would use a list for the open
vertices, and another list for the closed ones. The vertex to be expanded would be taken from
the list of open vertices (at the beginning of the list), and after expansion, it would be moved
into the list of closed vertices. The new vertices created during the expansion would be inserted
into the list of open vertices.
How to appl y cycle detection?
If we only perform the scanning on the current branch, then this can be easily realized,
because of the parent pointers.
If we chose the scanning of the whole database (i.e., we are also looking for multiple
paths), then our task is more difficult, since we have to traverse the whole tree, which is
impossible due to the parent pointers. But if we store the vertices in the above-mentioned two
lists, then this kind of cycle detection can easily be performed: we need to scan both the list of
the open vertices and the list of the closed ones.
4.3.2 SYSTEMATIC TREE SEARCH
On page 21, we can see a classification of problem-solving methods that differentiate systematic
methods and heuristic methods. In this chapter, we get to know the systematic versions of tree
search methods.
4.3.2.1 BREADTH-FIRST SEARCH
The breadth-first method always expands an open vertex with the smallest depth (if there are more
than one such vertices, it selects randomly). By this method, the different levels (vertices with the
same depth) of the tree are created breadth-wise, i.e., only after a level has fully been created we go
on with the next level. This is where the method's name comes from.
For the exact description, we assign a so-called depth number to each of the vertices stored in the
database. The breadth-first method selects the vertex with the smallest depth number to expand.
Definition 8. The depth number of a vertex in a search tree is defined the following way:
▫ g( s)=0 , where s is the initial vertex.
▫ g( m)=g (n)+1 , where m is a child of the vertex n .
It can be easily seen that, if the breadth-first method finds a solution, it is the optimal solution. This,
of course, has a cost: all the levels of the tree must be generated in breadth-wise, which means a lot
of vertices in case of certain problems. In practice, it causes difficulties when the problem to be
solved has long solutions, since finding them can be extremely time-consuming.
Completeness:
▫ If there is a solution, the method finds it in any state-space graph.
▫ If there is no solution, the method realizes this fact in the case of a finite state-space graph.
Optimality: finding the optimal solution is guaranteed.
Testing: can be performed sooner.
Although the breadth-first method finds a solution in finite steps without any cycle detection
techniques (assuming that there is a solution), in case of certain problems one of the extra tests from
Chapter 4.3.1 should be added. Naturally, this is worth only in connection with problems with
frequent cycles (and multiple paths) in their state-space graphs, since we can substantially lower the
number of vertices that are inserted into the database. Not to speak of the fact that we can also
realize in finite steps if there is no solution.
IMPLEMENTATION QUESTIONS
How to select the open vertex with the smallest depth number?
One option is to store the vertex's depth number in itself, and, before every expansion, to look
for the vertex with the smallest such number in the list of open vertices.
Another opportunity for this is to store the open vertices ordered by their depth number in a
list. The cheapest way of guaranteeing ordering is to insert each new vertex at the correct
position (by its depth number) into the already-sorted list.
It's easy to notice that, in this way, new vertices will always get to end of the list. So, the list of
open vertices functions as a data structure where the elements get in at the end and leave at the
front (when they are being expanded). I.e., the most simple data structure to store open vertices
in is a queue.
EXAMPLE
In Figure 16, a search tree generated by the breadth-first method can be seen, for a few steps, in the
case of the 3-mugs problem. It's interesting to follow on Figure 2 how this search tree look like
according to the state-space graph. In the search tree, we illustrate the open vertices with ellipses,
and the closed vertices with rectangles. The search tree given here has been built by the breadth-first
method without either cycle or multiple path detection. The red edges would be eliminated by a
cycle detection technique, which filters the duplications on branches. The yellow edges are the ones
(besides the red ones) that would be eliminated by a complete detection of cycles and multiple paths
(namely, by scanning over the database). It can be seen that such a cycle detection technique
reduces the size of the database radically, or at least it does in the case of the 3-mugs problem.
4.3.2.2 DEPTH-FIRST SEARCH
The depth-first search method expands an open vertex with the highest depth number(if there are
more than one such vertices, it selects randomly). The result of this method is that we don't need to
generate the tree's levels breadth-wise, thus, we may quickly find goal vertices even if they are deep
in the tree. This quickness, of course, has a price: there's no guarantee that the found solution is
Figure 16. The 3 Jugs problem solved by the breadth-first method.
optimal.
The depth-first method, compared to the breadth-first method, usually find a solution sooner, but it
also depends on the state-space graph. If the graph contains many goal vertices, or they are deep in
the tree, then the depth-first method is the better choice; if the goal vertices are rare and their depths
is small, then the breadth-first method is better. If our goal is to find an optimal solution, depth-first
search is (generally) out of the question.
Completeness:
▫ If there is a solution, the method finds it in a finite state-space graph.
▫ If there is no solution, the method realizes this fact in the case of a finite state-space graph.
Optimality: finding the optimal solution is not guaranteed.
Testing: can be performed sooner.
The similarity between the depth-first method and the backtrack method is striking. Both explore
the state-space graph „in depth”, both are complete if the graph is finite (if we don't use a cycle
detection technique, of course) and none of them is for finding the optimal solution. The question is:
what extra service does the depth searcher offer compared to the backtrack method? What do we
win by not just storing the path from the initial vertex to the current vertex in the database, but by
storing all the previously generated vertices? The answer is simple: we can perform a multiple path
detection. So, by combining the depth-first method with the cycle and multiple path detection
techniques detailed in Chapter 4.3.1 (namely by scanning the database before inserting a new
vertex) the method “runs into” a state only once during the search. Such a search is impossible with
the backtrack method.
IMPLEMENTATION QUESTIONS
How to select the open vertex with the highest depths number?
While in the case of breadth-first search, open vertices are stored in a queue, in the case of
depth-first search it's practical to store them in a stack.
EXAMPLE
In Figure 17, we introduce the search tree of the 3 jugs problem generated by the depth-first
method. Since the state-space graph of this problem is not finite (there are cycles in it), using some
kind of a cycle detection technique is a must. The vertices excluded by cycle detection are red,
while the vertices excluded by multiple path detection are yellow.
4.3.2.3 UNIFORM-COST SEARCH
The uniform-cost search is used for problems whose operators has some kind of cost assigned to.
Let's recall the concepts and notations from Page 11: the cost of an operator o applied to a state a
is denoted by
cost
o
(a)
, and the cost of a solution equals to the sum of the costs of all the operators
contained by the solution.
In case of the breadth-first and the depth-first methods, we did not assign a cost to operators.
Naturally, not all problems are such, that is why we need the uniform-cost method. To describe the
search strategy of the uniform-cost method, we have to introduce the following concept (like the
depth number defined on Page 37):
Definition 9. The cost of a vertex in the search tree is defined as follows:
▫ g( s)=0 , where s is the initial vertex.
▫ g( m)=g(n)+cost
o
(n) , where we have got m by applying the operator o operator to n .
The uniform-cost method always extends the open vertex with the lowest cost.
Notice that the depth number used by the breadth-first and depth-first methods is actually a special
vertex cost, in the case when
cost
o
(n)=1
for every operator o and every state n . So, the
Figure 17. The 3 Jugs problem solved by the depth-first method.
following fact can be appointed: the breadth-first method is a special uniform-cost method, where
the cost of every operator is one unit.
The properties of the uniform-cost method:
Completeness:
▫ If there is a solution, the method finds it in any state-space graph.
▫ If there is no solution, the method realizes this fact in the case of a finite state-space graph.
Optimality: finding the optimal solution is guaranteed.
Testing: can't be performed sooner, since such a modification endangers the finding of an
optimal solution.
In case of the breadth-first and depth-first methods, cycle detection was simple (if a vertex has
already got into our database, we haven't put it in again), but with the uniform-cost method, it gets
more complicated. Examine the situation when the vertex m is created as the child of the vertex n
, and we have to insert it into the database! Assume that m is already in the database! In this case,
there are two possibilities according to the relation between the cost of m (denoted by g( m) ) and
the cost of the new vertex (which is actually
g( n)+cost
o
(n)
):
▫ g( m)≤g(n)+cost
o
(n) : The database contains the new vertex with a more optimal path (the
cost is higher). In this case, the new vertex is not added to the database.
▫ g( m)>g(n)+cost
o
(n) : We managed to create m on a more optimal path. Then change the
vertex m stored in the database to the new vertex!
This last operation can be easily done: set m tot be chained to n , and update the cost assigned to
m (its g-value) to
g( n)+cost
o
(n)
.
A problem arises with such a chaining when m already has children in the tree, since, in this case,
the g-values of the vertices occurring in the subtree starting from m must be updated (since all of
their g-values originate in g( m) ). In other words, we have a problem when m is closed. This is
called the problem of closed vertices. Several solutions exist for the problem of closed vertices, but,
fortunately, we don't need any in case of the uniform-cost method, since the problem of the closed
vertices can't occur with the uniform-cost method. This is guaranteed by the following statement:
Statement 1. If a vertex m is closed in the database of the uniform-cost method, then g( m) is
optimal.
Proof. If m is closed, then we expand it sooner than the currently expanded vertex n . So,
g(m)≤g(n)
-
g(m)g(n)+cost
o
(n)
Namely, any newly explored path to m is more costly than the one in the database.
IMPLEMENTATION QUESTIONS
How to select the open vertex with the lowest cost ?
Unfortunately, in case of the uniform-cost method, neither a stack nor a queue can be used to
store open vertices. The only solution is to register the costs within the vertices, and to sort the
list of open vertices by cost. As we wrote earlier, the sorting of the database should be
guaranteed by insertion sort.
4.3.3 HEURISTIC TREE SEARCH
The uniform-cost method and the breadth-first method (which is a special uniform-cost method)
have very advantageous features, including the most important one, namely the ability to guarantee
optimal solution. But this has a heavy cost, which is the long execution time. This is caused by the
large database. The search tree stored in the database can consist of a lot of vertices. For example, if
a vertex in the tree has d children at most, and the length of the optimal solution is n , then the
search tree consists of
1+d+d
2
+d
3
+.+d
n
=
d
n+1
−1
d−1
vertices at most. Namely, the number of the vertices in the search tree is exponential in the length of
the solution.
The uniform-cost method is a so-called systematic method, that is it uses some „blind” search
strategy, and this causes the necessity of generating so many vertices. To avoid this, we try to
enhance our searchers with some kind of foresight, to wire our own human intuition up in the
method. The tool for this purpose is the so-called heuristics. Heuristics is actually an estimation,
that tells is which child of a vertex should we select to go on with the search. Notice that it is really
an estimation, since the subtree starting from the child has not been yet generated, so we can't be
sure if we are going to the right way (towards a goal vertex) in the tree
Anyway, it would be the best for us to have such a heuristics, that could exactly tell us in every
vertex which way (which child) leads us to the closest goal vertex. Such a heuristics is called
perfect heuristics and can generate
1+d+d +.+d = 1+n⋅d
vertices at most, so in the case of such a heuristics, the number of vertices is only linear in the
length of the solution. Of course, perfect heuristics does not exist, since if it existed then we could
know the solution in advance, so why bother with searching for it?!
The definition of heuristics is quite simple, as we have already introduced it in Chapter 4.1.3, in
connection with the hill climbing method. The point is that the heuristics as a function assigns a
natural number to every state. With this number we estimate the total cost of the operators that we
need to use to get from a given state to a goal state. If we think of a search tree: the heuristic value
of a vertex approximately gives the cost of a path to one of the goal vertices. In a state-space graph,
where the edges have the same cost (like in the case of breadth-first search), the heuristic value of a
vertex estimates the number of edges that leads into one of the goal vertices. Anyhow, by using
heuristics, we hope to reduce the size of the search tree.
4.3.3.1 BEST-FIRST SEARCH
The best-first method is a tree search method which selects the open vertex with the lowest heuristic
value to be expanded. The advantage of the resulting method is its simplicity, but, in the same time,
it loses on important features as compared to the uniform-cost (and the breadth-first) method.
Completeness:
▫ If there is a solution, the method finds it in any state-space graph.
Exception: if the heuristic value of every state is the same (1 unit).
▫ If there's no solution, the method recognizes this fact in the case of a finite state-space graph.
Optimality: finding the optimal solution is not guaranteed.
Testing: can be performed sooner, since we can't expect generating an optimal solution.
IMPLEMENTATION QUESTIONS
How to select the open vertex with the lowest heuristic value?
Obviously, the list of open vertices have to be sorted by the heuristic values of the vertices.
4.3.3.2 THE A ALGORITHM
In connection with the best-first method, we could see that using heuristics ruined all the good
features of the uniform-cost (and the breadth-first) method. However, we need to use heuristics to
solve the efficiency problems of the uniform-cost method. The uniform-cost method isn't effective
since it only considers the „past” during the search. The best-first method, on the other hand, only
„gazes into the future”, not learning from the past of the search, hence it doesn't always walks on
the logical way.
The idea: Combine the uniform-cost method with the best-first method! The resulting method is
called the A algorithm. To describe the search strategy of the A algorithm, we need to introduce the
following concept:
Definition 10. The total cost of a vertex n is denoted by f (n) , and defined as follows:
f (n)=g(n)+h(n) .
The A algorithm always selects the open vertex with the lowest total cost (f-value) to expand.
The f-value of a vertex n is actually the estimated cost of a path leading
from the initial vertex to a goal vertex via n (namely a solution via n ).
Notice that the uniform-cost method is a special A algorithm where the
heuristic value is zero for every vertex. This, of course, means that the
breadth-first method is a special A algorithm, too, since the costs of the
operators are equal and the heuristics is constant zero.
But there is a very sensitive point in the A algorithm: the cycle detection.
The same can be told as with the uniform-cost method (c.f. Chapter 4.3.2.3):
If the vertex we would like to insert is already in the database, and now it is
created with a lower cost, we have to replace the vertex in the database with
the new one (update its parent and g-value)! However, the problem of closed vertices could not
occur in the case of the uniform-cost method, but it can happen in the A algorithm. Namely, it can
happen that the vertex we want to replace ( m ) is closed, so it has descendants in the database,
which means that we have to update their g-values, too. The possible solutions for this problems
are:
(1) Traverse the subtree starting from m in the database, and update the g-values of the
vertices in it! Since we are only using parent pointers due to implementation questions,
such a traverse is not possible!
(2) Delegate the updating of the values to the A algorithm! We can force this by reclassifying
m as an open vertex. This means that every descendant of m (all the vertices of the
subtree starting from m ) will be regenerated once again with a lower g-value than their
current one, so their g-values will be updated by the A algorithm, too.
(3) Avoid the occurrence of the problem of closed vertices! This problem doesn't appear at all
in the uniform-cost method, so – as the uniform-cost method is a special A algorithm – the
question is what kind of heuristics is needed to completely avoid the problem of closed
vertices?
Option (3) will be examined in Chapter 4.3.3.4. For now, we use option (2)! So we allow the
algorithm to reclassify closed vertices to open vertices in certain cases. However, this brings forth
the danger of a never-stopping algorithm, since we don't know how many times (or even infinitely)
a vertex will be reclassified. Fortunately, the following statement can be proved:
Statement 2. Any vertex of the search tree will be finitely reclassified to open.
Proof. According to the definition (c.f. Page 11), we know that the cost of every operator is
positive. Denote the lowest such cost with 6 ! It can be easily seen that during reclassifying a
vertex to open, its g-value decreases at least with 6 . It's also obvious that every vertex's g-value
has a lower limit: the optimal cost of getting to the vertex (from the initial vertex). All of these
facts imply the truth of the statement.
Afterwards, let's examine the features of the A algorithm! By the previous statement, the following
facts can be told about the completeness of the A algorithm:
Completeness:
▫ If there is a solution, the method finds it in any state-space graph.
▫ If there is no solution, the method recognizes this fact in the case of a finite state-space graph.
But what about the optimality of the generated solution? We wonder if the A algorithm really
guarantees the generation of an optimal solution? A counterexample in Figure 18 shows that it does
not. On the left side of the figure, a state-space graph can be seen, in which we put the heuristic
values of the vertices aside them: for example, 5 for the initial vertex s , or 0 for the goal vertex
c (absolutely correctly, since all goal vertices should have a heuristic value of 0 ). We put the cost
of the edges (as operators) on the edges. In the figure, the optimal solution is depicted bold.
On the right side of the figure, we followed the operation of the A algorithm in this state-space
graph step by step. We put their g-values and h-values (cost and heuristics) aside every vertex. In
the 2
nd
step, it can be clearly seen that the method expands the open vertex a , since its f-value is
2+3=5 , which is lower than the other vertex's f-value of 3+4=7 .
The search finishes in the 3
rd
step, as the method selects the vertex c with its f-value of 6+0=6 ,
and it is a goal vertex. Notice that, by this way, the method has found a solution with a cost of 6 ,
which is not an optimal solution!
So the following facts can be told about the optimality (and testing) of the A algorithm:
Figure 18. Counterexample for the optimality of the A algorithm.
Optimality: generating the optimal solution is not guaranteed.
Testing: can be performed sooner, since the optimal solution is not guaranteed.
IMPLEMENTATION QUESTIONS
How to select the vertex with the l owest total cost ?
Similarly to the uniform-cost method, the costs are stored within the vertices, and their f-values
originate in these values. The list of open vertices is sorted by f-value.
How to handle the problem of closed vertices?
It's worth to select the 2
nd
one from the options described on Page 43. When inserting a vertex
into the database with a lowest cost than the existing one, it's worth deleting the old (closed)
vertex, and when it's done, the new (open) vertex can be inserted.
EXAMPLE
In the Figures 19 and 20, a search tree generated by the A algorithm for the Towers of Hanoi
problem can be seen, step by step, until finding a solution. We have decided to use the following
heuristics:
h(a
1
, a
2
, a
3
)=28−index( a
1
)−4⋅index (a
2
)−9⋅index (a
3
)
where
index( a
i
) =
¦
0 , if a
i
=P
1 , if a
i
=Q
2 , if a
i
=R
So, we assign an ordinal number to the rods: 0 to P , 1 to Q , and 2 to R . It can be seen that
the bigger discs are weighted more in the heuristics, as the goal is to move the bigger discs to the
rod R . It can also be seen that the subtraction from 28 happens to achieve the heuristic value of
0 in the goal state ( R, R , R) . Let us note that, in case of the Towers of Hanoi state-space
representation, it would have been more practical to mark the discs with numeric values (0,1, and 2)
from the beginning.
In the 3
rd
step, we indicated two such vertices which the A algorithm does not add to the database by
default, since their f-values are not lower than the f-value of the ones with the same content already
in the database. During the further steps, we do not show the other such vertices that are excluded
for the same reason.
0Figure 19. The A algorithm with the Towers of Hanoi (part 1).
The following conclusions can be drawn:
▫ The method founds an optimal solution. The question is if it happened by accident, or the chosen
heuristics forced the method to do so? C.f. the next chapter!
▫ The problem of closed vertices did not occur during the search. Namely, there wasn't any case
when a newly created vertex was found in the database as a closed vertex with a higher f-value.
▫ The use of heuristics made the search very insinuating. We went for a non-optimal direction only
once during the search: in the 6
th
step, when the vertex ( R, Q , R) was being expanded. But one
step later, the method returned to the optimal path.
4.3.3.3 THE A* ALGORITHM
The A* algorithm is a variant of the A algorithm that guarantees the generation of an optimal
solution. We surely know that the uniform-cost method is such a method, and it is obvious that this
advantageous feature is due to the constant zero heuristics. The question is: what kind of (but not a
constant zero) heuristics could guarantee the generation of an optimal solution?
To describe this accurately, we have to introduce the following notations for every vertex n of the
search tree:
▫ h*(n) : the optimal cost of getting to a goal vertex from n .
▫ g *(n) : the optimal cost of getting to n from the initial vertex.
▫ f *( n)=g *(n)+h *( n) : Consequently, the optimal cost of getting to a goal vertex from the
initial vertex via n .
Definition 11. A heuristics h is an admissible heuristics if the following condition holds for
every state a :
Figure 20. The A algorithm with the Towers of Hanoi (part 2).
h(a)≤h*(a) .
The A* algorithm is an A algorithm with an admissible heuristics. In the followings, we prove that
the A* algorithm guarantees the generation of an optimal solution. For this, we will also use two
lemmas.
Lemma 1. If there is a solution, the following condition holds for the A* algorithm in any given
time: The optimal solution has an element among the open vertices.
Proof. This is a proof by complete induction. Denote the vertices occurs in the optimal solution as
follows:
( s=) n
1
, n
2
, . , n
r
(=c)
▫ Before the 1
st
expansion, the initial vertex s is open.
▫ Basis: Assume that before an expansion, the optimal solution has an element among the open
vertices, and the one with the smallest index of these is n
i
.
▫ Inductive step: Let's see the situation before the next expansion! There are two options:
▪ If we haven't expanded n
i
right now, then n
i
remains open.
▪ If we have expanded n
i
right now, then n
i+1
has been inserted to the database as open.
Lemma 2. For every vertex n that was expanded by the A* algorithm:
f (n)≤ f *( s) .
Proof: According to the previous lemma, the optimal solution always has an element among the
open vertices. Denote the first such vertex with n
i
. Since n
i
is stored in the database on an
optimal path:
g( n
i
)=g *(n
i
)
Denote the vertex that is selected for expansion with n ! Since the A* algorithm always selects the
vertex with the lowest f-value for expansion, we know that f (n) is not higher than the f-value of
any open vertices, so in case of f (n
i
) :
f (n)≤ f ( n
i
)
Consequently,
f (n)≤ f ( n
i
) = g (n
i
)
_
=g * (n
i
)
+h(n
i
)
_
≤h*( n
i
)
≤ g *(n
i
)+h*(n
i
) = f *( n
i
) = f *( s)
The fact h(n
i
)≤h*( n
i
) originates in the admissible heuristics. The fact f *( n
i
)=f *( s) arises
from the knowledge, that n
i
is an element of the optimal solution, so the optimal cost of the
solutions going via
n
i
actually the cost of the optimal solution, namely f *( s) .
Theorem 1. The A* algorithm guarantees the generation of an optimal solution.
Proof: At the moment of the solution being generated, the algorithm selects the goal vertex
c to be expanded. According to the previous lemma:
f (c)≤f * ( s)
By considering the fact that c is a goal vertex:
f (c)=g(c)+h(c)
_
=0
= g (c) ≤ f *(s)
Thus, in the database, the cost of the path from the initial vertex to c (denoted by g( c) ) is not
greater than the cost of the optimal solution. Obviously, g( c) can't be less than the cost of the
optimal solution, thus:
g( c)= f *(s) .
Let us summarize the features of the A* algorithm!
Completeness:
▫ If there is a solution, the method finds it in any state-space graph.
▫ If there is no solution, the method recognizes this fact in the case of a finite state-space graph.
Optimality: generating the optimal solution is guaranteed.
Testing: can't be performed sooner.
4.3.3.4 THE MONOTONE A ALGORITHM
In the previous chapter we examined, what heuristics the A algorithm needs to guarantee the
generation of an optimal solution. Now, we examine what heuristics we need to avoid the problem
of closed vertices. The uniform-cost method (as a special A algorithm) avoids this problem with its
constant zero heuristics, but what can we say in general about the heuristics of the A algorithm?
Definition 12. A heuristics h is a monotone heuristics, if the following condition holds for
every state a : if we get the state a ' by applying an operator o to a , then
h(a) ≤ h( a' )+cost
o
( a)
.
The A algorithm with monotone heuristics is called the monotone A algorithm, and it can be
proved that the problem of closed vertices does not occur with this algorithm. This is the
consequence of the following theorem:
Theorem 2. For any vertex n that was selected by the A algorithm for expanding:
g( n)=g *(n) .
Proof: This is an indirect proof.
Assumption: g( n)>g *(n) . Namely, we assume that the algorithm hasn't found the optimal path
from the initial vertex to n .
Demote the vertices on the optimal path from the inital vertex to n as follows:
( s=) n
1
, n
2
, . , n
r
(=n)
Let n
i
be the first open vertex among these vertices. The following fact can be noted:
f (n
i
)=g(n
i
)+h( n
i
) = g *(n
i
)+h(n
i
)
Here we exploited that n
i
is already on the optimal path in the database; namely, g( n
i
)=g *(n
i
) .
Let us continue with the aforementioned formula!
f (n
i
)=g (n
i
)+h(n
i
) = g *( n
i
)+h(n
i
) ≤
≤ g *(n
i
)+h(n
i+1
)+cost
o
i
( n
i
) = g *(n
i +1
)+h(n
i +1
)
On the one hand, we exploited that the heuristics is monotone, thus,
h(n
i
)≤h( n
i+1
)+cost
o
i
(n
i
)
. On
the other hand, we exploited the fact
g *(n
i +1
)=g *(n
i
)+cost
o
i
(n
i
)
. Let us continue with the
aforementioned inequality in a similar way!
f ( n
i
)=g( n
i
)+h(n
i
) = g *(n
i
)+h( n
i
) ≤
≤ g *( n
i
)+h(n
i+1
)+cost
o
i
( n
i
) = g *(n
i +1
)+h(n
i+1
) ≤
≤ g *(n
i+1
)+h( n
i+2
)+cost
o
i+1
(n
i+1
) = g *(n
i+2
)+h(n
i +2
) ≤

≤ g *(n
r
)+h(n
r
)
So, where are we now?
f (n
i
) ≤ g *( n)+h(n) g (n)+h(n)= f ( n)
Here we used our indirect assumption, namely: g *(n)g (n) . Note that this leads to a
contradiction, since we have got that the f-value of n
i
is lower than the f-value of n , thus, should
have expanded n
i
instead of n !
The consequence of the previous two theorems is the following: in the monotone A algorithm a
cycle detection technique that is looking for the new vertex only among the open vertices is
absolutely sufficient. Accordingly, the following facts can be told about the completeness of the
monotone A algorithm:
Completeness:
▫ If there is a solution, the method finds it in any state-space graph.
▫ If there is no solution, the method recognizes this fact in the case of a finite state-space graph.
When summarizing the features of the monotone A algorithm, another main question is whether the
algorithm provides an optimal solution? Fortunately, it can be seen that the monotone A algorithm is
a special A* algorithm, so the answer to the previous question is „yes”. We prove this fact with the
help of the next theorem.
Theorem 3. If a heuristics is monotone then it's admissible, too.
Proof: It must be proved for any vertex n that the following fact holds in the case of the
monotone heuristics h :
h(n)≤h*( n)
There are two options. The first one is that no goal vertices can be reached from n . In this case
h*(n) equals to ∞ by definition, thus, the relation we want to prove obviously holds.
The other option is that a goal vertex can be reached from n . Take the optimal path among the
paths leading from n to any goal vertex, and denote its vertices as follows:
(n=)n
1
, n
2
, . , n
r
(=c)
By the monotonicity of h , the following relation can be drawn:
h(n
1
) ≤ h(n
2
)+cost
o
1
(n
1
)
h(n
2
) ≤ h(n
3
)+cost
o
2
(n
2
)

h(n
r−1
) ≤ h( n
r
)+cost
o
r −1
(n
r −1
)
Sum the left and right sides of these inequalities! We get the following fact:

i=1
r−1
h(n
i
) ≤

i =2
r
h( n
i
) +

i=1
r −1
cost
o
i
( n
i
)
If we subtract the first sum of the right side from both sides:
h(n
1
)−h(n
r
) ≤

i=1
r −1
cost
o
i
( n
i
) = h*(n)
Here, we exploited that h*(n) is actually the cost of the path (which is optimal!) leading to c
(namely to n
r
). Moreover, we even know that h(n
r
)=0 , since n
r
is a goal vertex.
Consequently:
h(n)≤h*( n)
Thus, the following facts can be told about the optimality and testing of the monotone A algorithm:
Optimality: generating the optimal solution is guaranteed.
Testing: can't be performed sooner.
4.3.3.5 THE CONNECTION AMONG THE DIFFERENT VARIANTS OF THE A ALGORITHM
We have got to know the following facts:
(1) The breadth-first method is a uniform-cost method, where the cost of the operators are one
unit, equally.
(2) The uniform-cost method is an A algorithm, where the heuristics is constant zero.
(3) The A* algorithm is an A algorithm, where the heuristics is admissible. The uniform-cost
method is an A* algorithm, too, since the constant zero heuristics is also admissible.
(4) The monotone Aalgorithm is an A algorithm, where the heuristics is monotone. The
uniform-cost method is a monotone A algorithm, too, since the constant zero
heuristics is monotone.
According to these facts, the appropriate relations among the different variants of the A algorithm
can be seen in Figure 21. As going top-down in the figure, we get more and more specialized
algorithms and more and more narrow algorithm classes.
5 2-PLAYER GAMES
Games – sometimes to a frightening amount – amuses the human intellect since the beginning of
civilization. Probably it comes from the aspect that a game can be understood as an idealized world
in which the opposing players compete with each other and face it, competition is present at every
field of life.
The first successes of artificial intelligence researches can be connected to games. Naturally, those
games are challenging to research, in which the players (either human or machine) have verifiable
influence on the outcome of the game. These games are called strategy games, like chess, draughts
or poker. There were gaming machines before the appearance of computer science, such as the
Spanish Quevedo's chess machine, which was specialized on the endgame of chess (the machine
had king and rook, while the human player had king) and was able to give a checkmate in any
situation (if the machine had the first step). The Nimitron, which was built in 1940 and was able to
win the Nim game any time, had similar qualities.
All these initial attempts remained isolated till the middle of the 1940s, when the first
programmable computers were developed. Neumann and Morgenstern's „Theory of Games and
Economic Behavior”, which is a basic work in this topic, came out in 1944 and gives an exhaustive
analysis of game strategies. Even in this book, the minimax algorithm plays and articular role and it
will become one of the basic algorithm in computer game programs.
The computer program, that was able to play through a full game of chess was created by Alan
Turing in 1951. Actually, Turing's program never run on a computer, it was tested with hand
simulation against a quite weak human opponent who defeated the program. The chess is a game
with huge state-space, this is the cause of the difficulties of creating a chess program. As time has
passed, the minimax algorithm was improved (in order to reduce the state-space) along certain
aspects – all of these, the alpha-beta purning, which was worked out by McCarthy in 1956, deserve
more attention.
When we try to transplant a game to computer, we have to give the following information in some
way:
▫ the possible states of the game
▫ the number of players
▫ the regular moves
▫ the initial state
▫ when will the game end and who will win (and how much)
▫ what information do the players have during the game
▫ if randomness has any part in the game
According to the above criterion, we can categorize the games as follows:
(1) By the number of players:
• 2-player games
• 3-player games
• etc.
(2) By the role of randomness
• Deterministic games: randomness plays no role
• Stochastic games: randomness plays role
(3) By finiteness
• Finite games: all players have finite moves and the game will end after finite moves.
(4) By the amount of information:
• Full information games: The players have all the information that arise during the
game. Card games are good counterexamples, as these have masked cards.
(5) By profit and loss:
• Zero-sum games: the total of the players' profit and loss is zero.
In the following, we will deal with 2-player, finite, deterministic, full information, zero-sum games.
5.1 STATE-SPACE REPRESENTATION
We can use the state-space representation to represent games. The state-space representation of a
game is formally the same as in Chapter 3.1. There are only a few differences in the content:
(1) The forms of states are (a , p) , where a is a state of the game and p is the player who is
next. The players will be marked with A and B , so p∈¦A, B¦ .
(2) In every goal state, it must be given exactly that who wins or loses in that given state or the
game is a drawn.
(3) Every operator o (invariably) has a precondition of the operator and a function of applying.
The function of applying shows if we apply o to a state (a , p) , than what state (a' , p' )
will we get. So it is important to define p' , which is the player who will move after player
p !
5.2 EXAMPLES
5.2.1 NIM
There are match-sticks in n pieces of mounds. The player with the next move can take any amount
of matches (at least one and no more than the number of matches in the mound) from a chosen
mound. The players take turns. Whoever takes the last match-stick loses the game.
Set of states: In the states, we store the number of matches in a mound. Let the state be
an n -tuple, where n>0 is a constant out of the state-space, denoting the number of the
mounds. A number max>0 must be also given in advance, which maximizes the number of
matches in the mound.
Naturally, the state must store the mark of the player with the next move.
So the set of states is the following:
A=
¦
(a
1,
a
2,
., a
n
, p)

∀ i 0≤a
i
≤max , p∈¦A ,B¦
¦
where every
a
i
is an integer.
Initial state: The initial state can be any state, so the only clause is the following:
k ∈A
Set of goal states: The game ends when all of the mounds are empty. The player who
takes the last match loses the game, so the player with the next move wins. Namely:
C=¦(0,.,0 , p)¦⊆A , p wins
Set of operators: The operators remove some matches from the mound. So the set of our
operators is:
O=
¦
remove
mound , pcs

1≤mound≤n , 1≤pcs≤max
¦
Precondition of the operators: Formalize when an operator
remove
mound , pcs can be
applied to a state
(a
1
,., a
n
, p)
! We need to formalize the following conditions:
▫ The mound
th
mound is not empty.
▫ The value of pcs is actually the number of matches the mound
th
mound has.
So, the precondition of the operator
remove
mound , pcs
to the state
(a
1
,., a
n
, p)
is:
a
mound
>0 ∧ pcs≤a
mound
Function of appl ying: Let's formalize what state
(a'
1
,., a'
n
, p' )
the operator
remove
mound , pcs
generates from the state
(a
1
,., a
n
, p)
! Namely:
remove
mound , pcs
(a
1
, ., a
n
, p)=(a '
1
,., a '
n
, p' )
where
a '
i
=
¦
a
i
−pcs , if i=mound
a
i
, otherwise
where 1≤i≤n
p' =
¦
A , if p=B
B , if p=A
NOTES
It's more worthwhile to denote the players with numeric values, where the most common solution is
using the values 1 and −1 . On one hand, this is quite useful in the function of applying of the
operator, e.g., the previous definition of p' can be as easily described as:
p' =−p
On the other hand, this notation of the players will come handy in the negamax algorithm
(Chapter 5.5).
5.2.2 TIC-TAC-TOE
We know this game as a 3x3 Gomoku (Five-in-a-row). On the 3x3 game board, the players put their
own marks in turns. The winner is who puts 3 of his or her marks in a row, in a column, or in a
diagonal line. The outcome of the game can be a drawn if the game board is full but no one has 3
marks abreast.
Set of states: In the states we need to store the whole 3x3 game boards, and of course the
mark of the player with the next move. One cell of the 3x3 matrix is 0 if none of the players
has put a mark there.
So the set of states is defined as follows:
A=
¦
(T
(3×3)
, p)

∀ i , j T
i , j
∈¦0, A ,B¦ , p∈¦A , B¦
¦ .
Initial state: In the initial state the game board is completely empty, and player
A
is going
to put his or her mark first!
k=
(
(
0 0 0
0 0 0
0 0 0
)
, A
)
Set of goal states: The game may end because of two reasons:
▫ One of the players has 3 marks abreast.
▫ The game board is full.
Namely:
C=C
1
∪C
2
, where
C
1
=
¦
(T , p)

∃i T
i ,1
=T
i ,2
=T
i ,3
=p'

∃i T
1, i
=T
2, i
=T
3,i
=p'

T
1,1
=T
2,2
=T
3,3
=p'

T
1,3
=T
2,2
=T
3,1
=p'
¦
, p' wins
(see the definition of p' in the operators' function of applying)
and
C
2
=
¦
(T , p)∉C
1

¬∃i , j T
i , j
=0
¦
, drawn
In the set
C
1
, we specify the winning goal states. As it was the opponent of p last time, it's
sufficient to look for triplets of the marks of p' on the board. In the condition, that consists of
four rows, we specify (in order) the triplets in one row, in one column, in the main diagonal ,
and in the side-diagonal.
In the set
C
2
, we give the drawn goal states. The game ends in a drawn if the board is full (and
none of the players has 3 marks abreast).
Set of operators: Our operators put the mark of the current player into one cell of the
board. So, the set of our operators is defined as follows:
O=
¦
put
x , y

1≤x , y≤3
¦
Precondition of the operators: Let's formalize when can we apply an operator
put
x , y
to a state (T , p) ! Naturally, when the cell ( x , y) of T is empty.
So, the precondition of the operator
put
x , y
to a state (T , p) is:
T
x , y
=0
Function of appl ying: We have to formalize what state
(T ' , p' )
the operator
put
x , y
generates from a state (T , p) ! So:
put
x , y
(T , p)=(T ' , p' )
where
T '
i , j
=
¦
p , if i=x ∧ j =y
T
i , j
, otherwise
where 1≤i , j ≤3
p' =
¦
A , if p=B
B , if p=A
5.3 GAME TREE AND STRATEGY
According to the state-space representation of the 2-player games, using the method given in
Chapter 3.2, we can create a state-space graph. As we told it earlier, we only deal with finite games,
so this graph must be finite, too. This also indicates that the graph may not contain cycles. On the
other hand, cycles are usually eliminated by the game's rules. By unfolding the state-space graph
into a tree, we get the so-called game tree.
2
One play of the game is represented by one branch of
the tree.
For a game program to be able to decide the next move of a player, it needs a strategy. The strategy
is actually a kind of instruction: which operator should the computer apply to a given state? It would
be an interesting question whether does any of the players have any winning strategy, in the case
of a given game? Namely, such a strategy in which the operators are applied following the
instructions will always lead to the given player's victory (no matter what moves the
opponent takes).
For this, we need to investigate how a given strategy (in connection with either player A or B )
appears in the game tree. If we want to represent the possible strategies of the player
p∈¦A , B¦
,
then we transform the game tree into an AND/OR-graph in the following way: with an arc we
conjoin all the edges that start from a vertex in which the opponent of p moves. In Figure 23, we
can see a AND/OR-graph from the point of view of player A . We have conjoined the edges starting
from a vertex in which player
B
moves. We call the resulting conjoined edges an AND-edge
group, which represent the fact that player p has no influence on the opponent's moves, thus,
the strategy of p must be prepared for any of the opponent's move. The edges highlighted with red
in the figure could be the moves of a strategy of A , since they unambiguously define the moves of
A
during the game.
Now, let us formalize, in an exact way, what we mean by and AND/OR-graph and a strategy!
Definition 13. An AND/OR-graph is a ordered pair 〈 V , E〉 where
▫ V ≠∅ is the set of vertices, and
▫ E is the set of the hyperedges, where
E⊆¦(n , M)∈V ×P(V ) ∣ M≠∅¦
An (n , M)∈E hyperedge can be of two kinds:
▫ an OR-edge, if ∣M∣=1 .
2 Unfolding into a tree actually means the elimination of the multiple paths. So, the individual copies of the vertices
(and the subtrees starting from them) should be inserted into the tree.
Figure 23. The game tree as a AND/OR-graph.
▫ an AND-edge group, if ∣M∣>1 .
The hyperedge is the generalization of the edge concept as we know. A hyperedge can be drawn
from one vertex to any number of vertices; that's why we define M as the set of vertices
3
. The
usual edge concept is equivalent to the concept of an OR-edge, since it is a special hyperedge that
leads from one vertex to exactly one vertex.
We call the generalization of the corresponding path concept hyperpath: a hyperpath leading
from vertex n to the set M of vertices is a sequence of the hyperedges between n and M.
After all of the above, it is easy to formalize the strategy of player p : a hyperpath leading from the
initial vertex to a set M⊆C of vertices (namely, each element of the M is a goal vertex) in an
AND/OR-graph from the point of view of p . In Figure 24 and Figure 25, a possible strategy for
player A and player B can be seen, respectively, framed in the tree.
3 P(V ) denotes the powerset of the set V (namely, the set of all the subsets of V ).
Figure 24. A possible strategy for player A.
5.3.1 WINNING STRATEGY
One of the seducing goals in connection with a game is to look for winning strategies. Does one
such thing exist? If it does, then does it for which player?
First, we need to clarify what we call a winning strategy of player p : such a strategy of p which
leads (as a hyperpath) to such goal states in which p wins.
Theorem 4. In every game (examined by us) that cannot be a drawn, one of the players has a
winning strategy.
Proof: We generate the game tree, and then we label its leaves with A and B , depending on who
wins in the given vertex.
After this, we label the vertices in the tree bottom-up, in the following way: if p moves in the
given vertex, then
▫ it gets the label p if it has a child labelled with p ;
▫ it gets the label of the opponent, if it doesn't have a child labelled with p (so, every children
of the vertex is labelled with the opponent's mark).
After labelling all the vertices, the label of the initial vertex (i.e., the root) shows the player
which has a winning strategy.
In games which can end in a drawn, a non-losing strategy can be guaranteed for one of the players.
5.4 THE MINIMAX ALGORITHM
Since a game tree can be extremely huge, it is an irrational wish for our game program to wholly
generate it (and look for a winning strategy in it). The most we can do is that before every move of
the computer player, we generate a part of the game tree. This tree has the following properties:
▫ its root is the current state of the game, and
▫ its depths is limited by value maxDepths , that has been given in advance.
Generating the tree this way is the same as the common „trick” applied in reality, when the players
think ahead for a few moves, playing all the possible move combinations in their heads. The value
maxDepths specifies for how many moves the program should think ahead. It's worth asking for
the value of maxDepths as a difficulty level at the start of our game program.
In Figure 26, we built up an appropriate subtree for the Tic-tac-toe game, starting from the state
(
(
0 A A
0 B 0
A B 0
)
, B
)
, and for maxDepths=2 .
The leaves of the tree built in this way can be of two sorts. Some of them are goal vertices, and we have clear concepts about their usefulness: if the
given player wins in the leaf, then it's a „very good” goal state for him; if he loses, then it's a „very bad” one.
The other leaves are not goal vertices; we haven't expanded them because of the depth limit (most of the leaves in Figure 26 are such leaves). That's
why we are trying to define the „goodness” of leaves with some kind of estimation; so, we need a heuristics.
HEURISTICS
How to specify a chosen heuristics? The definition of heuristics introduced in Page 24 must be reconsidered in connection with 2-player games:
▫ Heuristic values must be arbitrary integer values. The better it is for the given player, the greater (positive) the value is, and the worse it is for the
player, the lower (negative) the value is.
▫ In case of a goal state, the heuristic value should be 0 only if the goal state results in a drawn. Memo: in Page 24, we assigned 0 to every goal
states. In the case of 2-player games, if the given player wins in the goal state, the heuristic value should be a very great value, while if the player
loses, it should be a very small value.
▫ Since there are two players, we need two heuristics.
h
A
is the heuristics for player A , while
h
B
is the heuristics for player B .
In a given game, we define the heuristics. For example, in the case of Tic-tac-toe, let the heuristics h
p
(where p∈¦A , B¦ ) be defined as follows:
60
(1) 10 , if we are in a goal state, where p wins.
(2) −10 , if we are in a goal state, where the opponent of p wins.
(3) 0 , if the goal state is a drawn.
(4) The number of all rows, columns, and diagonals that p „blocks” (his mark occurs there).
THE FUNCTIONING OF THE MINIMAX ALGORITHM
Denote the player in the root of the tree (who has the current move) with aktP ! Let us assign a
weight to all of the vertices (a , p) of the tree, by the following way:
(1) If (a , p) is a leaf, then let its weight be
h
aktP
(a , p)
.
(2) If (a , p) is not a leaf, it means that it has some children. The weights assigned of its
children are used to calculate the weight of their parent. This is done as follows:
• If p=aktP , then the maximum of the children's weights is assigned to (a , p) .
• If p≠aktP , then the minimum of the children's weights is assigned to (a , p) .
The explanation of calculating maximum/minimum is obvious: aktP is always about to move
towards the most advantageous state, that is, the state with the highest heuristic value. The opponent
of aktP is about to do quite the opposite.
The values besides the vertices in Figure 26 are the weights of the vertices, according to the
heuristics of player aktP=B , who takes his move in the root.
After assigning a weight to each vertex in the tree, comes the step why we have arranged all these
before. We determine which operator is worth applying in the root of the tree (that is, to the current
state of the game). Namely: the game program should use (or suggest, in the case of the human
player's move) the operator that (as an edge) leads from the root of the tree to the child with the
highest weight. In Figure 26, the operator
put
1,1
is being applied (or advised) by the game
program.
5.5 THE NEGAMAX ALGORITHM
The difficulty of using the Minimax algorithm is that two kinds of heuristics must be defined for a
game. It's easy to see that there is a strong relation between the two heuristics. This connection can
be described with simple words, as follows: the better a state of the game is for one player, the
worse it is for the other player. So, the heuristics for one player can be simply the opposite (negated)
of the heuristics for the other player.
Consequently, only one heuristic is absolutely sufficient. Let the heuristic value of the state (a , p)
be calculate by the heuristics for p .
Starting on this track, we can make the minimax algorithm simpler at two points:
(1) When assigning heuristic values to the leaf vertices of the generated tree, we don't need to
take into account which player moves in the root of the tree.
(2) When calculating the weight of a non-leaf vertex (a , p) , we calculate maximum, in all
cases. The trick is to find a way to transform the weight of the child (a' , p' ) of a vertex
(a , p) , as follows:
• If p≠p' , then we use the negated value of the weight of (a' , p' ) to determine the
weight of (a , p) .
• If p=p' , than we use the (unchanged) weight of (a' , p' ) to determine the weight of
(a , p) .
In Figure 27, it can be seen the tree of the Tic-tac-toe game, generated by the Negamax algorithm,
this time for maxDepth=3 . It is worth contrasting with Figure 26. In these figures, it can also be
seen that player B , who is the currently moving, makes a clever move as he or she is controlling
the game to the only one direction where he or she has any chance of winning. So, we've chosen the
heuristics quite well.
5.6 THE ALPHA-BETA PRUNING
If we choose a high value for maxDepth , then the generated tree (and also, the time-demand of the minimax algorithm) will be very large. So, it is
worth optimizing the algorithm, based on the observation that it is often unnecessary to generate certain parts of the tree. Eliminating these parts is the
main goal of the alpha-beta pruning.
63
With the alpha-beta pruning, we can interrupt the expanding of a certain vertex. We do this in cases
when it turns out before completing the expansion that the weight of the vertex being currently
expanded will have no effect on the weight of it's parent. Let's see two possible situations:
(1) Al pha cut-off: We are calculating minimum in the current vertex, and maximum in it's
parent. We interrupt the expansing of the current vertex, if the minimum value calculated in
the vertex becomes less then the maximum value calculated in it's parent. We can see such a
situation in Figure 28: the vertex being expanded bears the caption ≤4 , since we have
already created a child with a weight of 4 , hence, the weight of the current vertex will be
not greater than 4 . For similar reasons, its parent's weight will be not less than 9 , so we
don't need to generate all the other children of the current vertex, since the weight of the
current vertex is irrelevant from this point on.
(2) Beta cut-off : It is the same as the alpha cut-off, but in this case, we calculate maximum
in the vertex being currently expanded, and calculate minimum in it's parent. We can see
such a situation in Figure 29.
In Figure 30, a tree generated by the Minimax algorithm for Tic-tac-toe can be seen, for
maxDepth=3 . The parts of the tree that the algorithm doesn't need to create due to the Alpha-beta
pruning are in a red frame. Count them, and it will turn out that we need to generate 20 vertices less,
and only need to generate 15 vertices. We managed to cut 57% of the tree.
Figure 28. The Alpha cut-off.
Figure 29. The Beta cut-off.
65
6 USING ARTIFICIAL INTELLIGENCE IN EDUCATION
The main topic of artificial intelligence consists of the solution searcher algorithms, that search for
the path from the starting vertex to the goal vertex in a graph. It is difficult to represent these
algorithms on a (white)board, as the state of a vertex is constantly changing. For example, in case of
the depth-first search, a vertex can be unexplored, open or closed. As there is no space on the board
to draw the graph many times, we can only represent what we want by re-drawing the status of the
vertex in the same graph. By this way, the audience easily loses the time-line sequence. We are
looking for a solution for this problem among others.
One suggested solution is to use animated figures, where the algorithm and the graph can be seen.
On the right side of the figure, the execution of the algorithm can be seen step-by-step. The actual
line is coloured. If any command changes the state of the vertex, than the graph's appropriate vertex
changes on the left of the figure. Usually it turns to another colour. Of course, every colour's
meaning is listed.
According to our experiences, the use of this aid helps and deepens the understanding of the
searcher algorithms.
We noticed that the students are afraid of the searcher algorithm as they find them to complicated,
that exceed their abilities. So it's very important to make these algorithms tangible and bring them
close. One great method for this is the visualization of the algorithm, that's why our digital notes
includes several animations.
6.1 THE PROBLEM
The problem has two levels, one that is psychological and another that is technical.
When realizing the searcher algorithms, the students first face the problem that they need to use
their programming knowledge in a complex way. First, they need to create a state-space
representation along with the attached data structures (usually with the help of a State and Vertex
class), than they need to select the appropriate algorithm which is recursive in many cases.
As the length of the solution is not determinable beforehand, it's storing requires a dynamic data
structure, which is a usually a list (or a stack, or maybe a queue). So to summarize, they need to
routinely use:
▫ the classes,
▫ the recursion and
▫ the dynamic data structures.
These requirements scare off many students of the subject before they could see it's beauty. If the
student feels that a subject is to complicated, it exceeds his or her abilities than he or she secludes
himself of understanding it. This is the psychological problem.
The other problem is rather technical. Representing the main solution searcher algorithms on the
board is uneasy, as the states of the vertices are changing with time. For example, in case of the
depth-first search, a vertex can be unexplored, open or closed. As there is no space on the board to
draw the graph many times, we can only represent what we want by re-drawing the status of the
vertex in the same graph. By this way, the audience easily loses the time-line sequence. We find a
solution to both problems if we call for the aid of multimedia.
Solution suggestion
Using multimedia in education is not a new thing. It's a widely accepted view that the materials
thought in any subject is more easily learnt if we use multimedia aids during the teaching, such as:
col ourful figures,
vi deos, animations,
interacti ve programs.
The cause of this phenomenon is that multimedia bring closer and makes tangible anything that was
said before in theory.
At the same time, using multimedia has drawbacks too. It's usage needs such tools that are not
always given in a lecture room, they may break down, their starting takes time and such problems.
The practical part of teaching artificial intelligence usually takes place in a computer laboratory, so
the conditions of multimedia are usually given.
The representable information changes in the cases of the different solution searcher algorithms, so
it's required to define for each of them what information should appear in the animation, as part of
the graph or completing it and what kind of data structure is used as a database to store the graph. In
the following, we will discuss the differences between the representation of the different searcher
types.
6.1.1 NON-MODIFIABLE SEARCHERS
We use the non-modifiable searchers in such special cases when our goal is not to produce a
sequence of operators that lead to the solution, but we want to know if there is a solution for the
problem, and if the answer is yes, than we have to produce this goal state. One of the most well-
know and most customizable type is the Mountaineer method. This is a heuristics searcher, which
means that the algorithm selects on operator for a given state by an estimation given by us. This is
very fortunate in the cases when we are able to give an effective heuristics. In an opposing case, we
can use the more simple Trials and Errors method which selects operators randomly. In the notes,
we demonstrate the build-up of the animation on the Mountaineer with restart method. In this
version we record the list of states that we couldn't use to continue the search and we have to give
the maximum number of members of this list. When we are unable to continue the search, the
current vertex is inserted into the list of forbidden vertices and the search restarts.
On the figure above we can see the searcher in a state when we have inserted the D vertex to the list
of forbidden vertices after a restart. The searcher's current, actual vertex is C. The next step will be
done from this point and will select the operator leading to vertex E. We know this because the
heuristics of the state in the E vertex is smaller. The heuristics of the different states are marked
with the number by the vertex.
Legend:
Notions Markings
Not yet visited vertex blue
Current vertex Deep green
Visited vertex green
Forbidden vertex/state red
Heuristics Number by the vertex
Starting vertex There's an S by the vertex
Terminal vertex There's a T by the vertex
List of forbidden states In the yellow list under the caption
We do not note the number of restarts separately, if it is necessary, we portray it when restarting.
6.1.2 BACKTRACK SEARCHERS
One type of the modifiable searchers is the backtrack searcher, where we expend our database
outside the current vertex with the list vertices leading to the actual vertex. By this, we have the
option to continue the search to another direction if the searcher runs into a dead end. This backstep
gives the name of the searcher. If we find a terminal vertex, by the expanded database we have the
option to give the sequence of operators that we use on the initial state to get the goal state. On the
figure, you can see the backtrack searcher with length-limit. In this version, we work with a
database of which size is given beforehand. We can use this if we can estimate the step number of
the optimal solution well.
On the figure above we can see the searcher in a state when the database that is used to store the
vertices gets full. The database on the figure gets filled with elements from the bottom to the top.
This also represents well the operation of the stack data structure, that is usually used for realizing
the database in case of the backtrack searchers. We can easily read from the figure that as the
current vertex is the D and it is located under B, we will step back to vertex B in the next step.
Legend:
Notions Markings
Not yet visited vertex blue
Current vertex Deep green
Visited vertex green
Operator, that we use to step forth Number by the arrow
Starting vertex There's an S by the vertex
Terminal vertex There's a T by the vertex
The stack representing the database The column right to the graph
6.1.3 TREE SEARCH METHODS
These make up the other great group of the modifiable solution searchers. By further extending the
database we try to find the solution faster and if it's possible to find the solution with the least steps.
While we stored one path in the case of the backtrack searchers, we store more paths at the same
time in case of the tree search methods, that can be represented as a tree. We do the search on
different paths in turns. The different algorithms differ in the method they use to select how to
continue the search. Going forth in the tree is extending (or extension), by this we differentiate
between open and closed vertices. The closed vertices are the ones we already extended, the others
are considered open vertices. Another new notion is the depth number. A vertex's depth number is
given by the steps we need to reach it from the starting vertex.
On the figure above we can see the depth-first method which selects the vertex with the lowest
depth number for extending. If more vertices have the same depth, it selects the next vertex for
extending randomly or by agreement, like from left to right. The searcher on the figure is currently
at extending the G vertex. In the next step, the G vertex will be closed and the K vertex will appear
in the list of open vertices and it's colour will change to green too.
Legend:
Notions Markings
Not yet visited vertex blue
Vertex under extension Deep green
Open vertex green
Closed vertex red
Depth number Number by the vertex
Starting vertex There's an S by the vertex
Terminal vertex There's a T by the vertex
List of open/closed vertices The list under the given caption
6.1.4 DEPTH-FIRST METHOD
We show the depth-first method in short here.
It is visible that the figure has two main parts. On the left part of the animation we can see a graph
in which we are searching for the solution. On the right side, we can examine the algorithm of the
depth-first method itself. On the right side of the figure we can see as the algorithm is executed step
by step. The actual line is red. If any command changes the state of a vertex, then on the left side of
the figure, the appropriate vertex of the graph changes. The yellow vertices mark the not yet visited
vertices, a the blacks are the closed ones and the red ones are the open vertices. The starting vertex
is the uppermost vertex of the graph, the goal (or terminal) vertex is at the bottom. The numbers in
the vertices show the depth number of the vertices.
The important fact, that the formation of the graph on the picture or in the animation is not entirely
random also have to be mentioned, indeed, it was built that way willingly. It's goal is for the figure
to show special cases that are usually problematic with some algorithm of artificial intelligence.
Such as the diamond and the loop.
To understand the animation, we summarize the depth-first method in short. The depth-first method
extends the vertex with the highest depth number (the number in the vertex) of the open vertices
(the red ones). If there are more such vertices, it selects randomly. We put a ring around the vertex
that is selected for extension. As the result of the extension, the not yet visited (yellow) children of
the vertex will be open (red) and the extended vertex will be closed (black). The search ends if we
are out of open vertices (than we have no solution), or if the last visited vertex is a goal vertex (in
this case we have a solution).
According to the above explanation, it's visible that such abstract notions, like open vertices can be
specified: red vertices. The table below pairs the notions and the markings of the animation:
Notions Marki ngs
Vertices not yet visited yellow
open vertices red
closed vertices black
Depth number the number in the vertex
vertex selected for extension ringed vertex
extension/extending the yellow off-springs turn red, the ringed ones turn
black
6.1.5 2-PLAYER GAME PROGRAMS
2-Player games, like the chess, the draughts or the tic-tac-toe can be represented with a game tree.
We can create this by using the game's state-space graph. In case of the 2-player games, each branch
of the tree represents a different play of the game. To decide in the certain states that where to step
next, namely which operator to use, we need to know which player can step in the turn. According
to this, we can built up the And/Or graph, which helps with the path selection.
We can see the Minimax algorithm on the figure above. The algorithm works by generating the
game tree from the current state for a given depth. When we are done with this, in the leaves of the
tree, namely in the finally generated vertices of the branch, we try to give an estimation about how
good the given state is for us. The children elements' minimum or maximum is inserted to the
vertex, heading for the current vertex up in the tree, depending on who has the next step, our
opponent or we. That what do we need to select and when is noted on the right side of the
animation, by the frame.
Legend:
Notions Markings
Current vertex Vertex with the "akt" caption
Selectable operators Caption by the arrows
Depth limit On the left ledge
Player On the right, marked with captions "A" and "B"
Minimum/maximum selection On the right, by the frame
6.2 ADVANTAGES AND DISADVANTAGES
According to the experiences, with the use of the markings of the animation, the explanation is
much more intuitive. The algorithm becomes more touchable and visible, so less will feel that it
doesn't worth paying attention, as the subject is too difficult. Hence, we have found a solution for
the psychological problem.
It is called psychological difference reduction when a problem that is too difficult to solve is traced
back to more simple problems that can be solved. This is the explanation why the use of animations
reduces the reluctance of the difficult subject – as it is easier to understand the animation.
The animation also solves a technical problem, as we don't need to use a board to draw a graph.
Everyone can see the animation on his/her own computer or it can be displayed with a projector,
and if the student was unable to follow it, it will restart automatically.
It is a disadvantage that we need a computer or a projector at least to see the animation. Without
electricity, these devices don't work. To download the animation, internet connection is required.
The other disadvantageous phenomenon is that the audience may recall the concepts with their
markings during the feedbacks. This is a greater problem, but if somebody reached the point to
understand the algorithm, he or she doesn't need to put so much more energy to use the proper
concepts. Moreover, during the feedbacks, the teacher can help in a mediate way with the markings,
if the student gets stuck in his/her train of thoughts.
7 SUMMARY
As a summary, let's see a short heuristics, what can be done if we start to solve a problem with the
techniques learnt here, but our program doesn't find the solution in time.
In our notes, we introduce the most well-known variations of the graph searcher algorithms, we
showed it how can these be used to solve problems that are difficult in a way and a human can't
solve them for the first time. At the same time, these algorithms are closer to systematic trials and
errors, except for a few trivial optimizations, like the loop-checking.
Systematic trials and errors, is that all? Yes, that's all! We have to try many possibilities, and have to
cover all the possible cases. This is systematicy. Of course, another question is how fast do we try
these possibilities.
Till a certain level, it makes no difference what order do we try the operators, as the state-space is
quite small and our fast computer can scan through fast. But we can easily run into a problem,
which has a very large state-space.
In such cases, we can still try to be tricky by optimizing the operators or the algorithms themselves,
for example by implementing the more effective relative of backtrack, the backjumping. It's even
more effective if we use a lazy data structure. This means that we postpone everything we can of the
algorithm's steps till we can. Such variants of the algorithms are called lazy algorithms and the data
structures that support them are called lazy data structures.
This helps, but it's not the answer of artificial intelligence for the problem. The artificial
intelligence's answer is the correct choice of the problem-representation. A problem can be
represented with a very large state-space and with a quite small too.
How do we know that if the state-space is small or large? Generally, the less operators we have the
smaller the state-space is, so we have to strive to have as few operators as possible.
Let's see a simple problem: In case of the 3 monks and 3 cannibals problem, one operator can be
that when the boat brings some monks and cannibals to the other side of the river and a different
operator can be when we sit in and get out. In the second case, the state-space will be much larger.
Unfortunately, after a difficulty level, not even creating a small state-space will help, as the solution
of the problem will still takes many hours. This is when the second super-weapon comes into view:
the use of heuristics.
What did we say? The graph search is only systematic tries and errors? Heuristics tell which
operator worth trying. A good heuristics can hasten up the search by many degrees.
In the notes, we haven't dealt much with heuristics, only in case of the Minimax algorithm. This is
by no means an accident. If the kind reader gets to write artificial intelligence for games that tells
what the computer's next step should be, he will have an easy time except for writing the heuristics.
This will be the hardest part and it's up to the heuristics how „smart” the game will be.
If the second super-weapon doesn't help either, than we still have a chance. We abandon systematicy
and trust in tries and errors. Yes, the last super-weapon is the mountaineer – method, the
mountaineer – method with restart. Have you ever thought?
On one hand, the mountaineer – method uses heuristics, on the other hand, it is really simple and in
this manner it is quick. This algorithm is unable to realize if there is no solution, but in many cases,
this is not important, the only thing that counts is to find the solution fast if it exists. If the heuristics
is good, we have a good chance for this. If we don't find the solution fast, we give up the search
after a given time.
We use the mountaineer – method with restart to solve one of the most significant artificial
intelligence problem, the SAT problem. The version we use there is called Random Walk. We
analyse the SAT problem is details in the Calculation Theories notes.
All kind readers can see that this short train of thoughts has just scratched the surface of artificial
intelligence, but at the same time all that is written here gives a strong base for further studies.
8 EXAMPLE PROGRAMS
In this chapter we review some very cohesive classes. The classes materialize two algorithms, the
backtrack searcher and the depth-first method. These classes embody the experience of a few years,
we have tried and tested them many times, so they probably work. However, at some places the
source must be altered if someone wants to use it for another graph search problem. Fortunately,
this only needs an own State class, which have to originate from the AbstractState class. Other than
this, neither the Vertex nor the BackTrack (and other) classes need rewriting, only the main
program.
In the next part of the chapter, we review the different classes emphasizing what needs rewriting
and what can stay in it's current form.
8.1 THE ABSTRACTSTATE CLASS
The AbstractState class is the ancestor of every State class. It is the same as the state-space
representation. It orders that each and every State class have to contain a IsState function, that
matches the State predicate, a IsGoalState function, that matches the goalstate predicate. The set of
operators is given in a bit oakwardly way for the first sight. There is no such set, but we know how
many member would it have. This is given by the NumberOfOps function. This must be extended
by the children too. We don't need to specify the operators to, but we know that we can access them
by the so-called super-operator. This is what the SuperOperator(int i) method is for.
The other methods don't need overwriting. The Clone method probably won't need rewriting in any
children class. It would only need it if the inner fields of State would be arrays. In such case, the
arrays need cloning too. We call this deep cloning.
The Equals method need rewriting if we want to use backtrack with loop-checking or depth-first
method with loop-checking (with depth-first method, the loop-checking is the default). If we
overwrite the Equals method than we should overwrite the GetHashCode method too. This is not
obligatory, but the compiler will give ugly warnings instead.
8.1.1 SOURCE CODE
/// <summary>
/// The ancestor of every state class.
/// </summary>
abstract class AbstractState : ICloneable
{
// Checks if the inner state is a state.
// If yes, then the result is true, else it's false.
public abstract bool IsState();
// Checks if the inner state is a goal state.
// If yes, then the result is true, else it's false.
public abstract bool IsGoalState();
// Gives the number of the basic operators.
public abstract int NumberOfOps();
// The super operator is used to access all the operators.
// Is true if the i
th
basic operator is executable for the inner state.
// It must be called from a "for" cycle from 0 to the number of basic operators. For example:
// for (int i = 0; i < state.GetNumberOfOps(); i++)
// {
// AbstractState clone=(AbstractState)state.Clone();
// if (clone.SuperOperator(i))
// {
// Console.WriteLine("The {1}th for the {0} state" +
// "operator is executable", state, i);
// }
// }
public abstract bool SuperOperator(int i);
// Clones. We need it because the effects of some operators must be withdrawn.
// The most simple is to clone the state. That is what we call the operator for.
// If there is a problem, we return to the original state.
// If there is no problem, it will the clone be the state of which we continue the search.
// This executes shallow cloning. If the cloning is swallow enough, than it does not need overwriting in the child
class.
// If deep cloning is required, than it must be overwritten.
public virtual object Clone() { return MemberwiseClone(); }
// Only needs overwriting if we use backtrack with memory or depth searcher.
// Else, this may remain the basic implementation.
// By programming technique, this is a hook method, that we can overwrite without violating the OCP.
public override bool Equals(Object a) { return false; }
// If the two instances are equal, than their hash codes are equal too.
// So, if the Equals method is overwritten, this should be overwritten too.
public override int GetHashCode() { return base.GetHashCode(); }
}
8.2 HOW TO CREATE MY OWN OPERATORS?
We talk about the BlindState class in this chapter. This class has no function, only show in general
how to create basic operators and operators with parameters and how to connect these in the super
operator. We call basic operators the operators with no parameters or with fix parameters. These
must be listed in a switch construct in the super operator. With an operator with parameters, more
basic operators can be given with different values of the parameters.
From the example, it is visible that every operator has a true/false returning value. It is true, if it is
executable on the current inner state of the State instance, else it is false. By programming
technique, the operators can be done in many ways. Maybe the one selected here follows most the
encapsulation theory of the OOP, which says that the inner state of the instance can only be
modified by the methods of the instance. We know of artificial intelligence that these methods are
called operators.
As the basic operators need to be accessed through the super operator, their visibility level can be
private. Only the super operator needs to be public.
Every operator has a precondition. They also have a postcondition, but that is the same as the
IsState predicate. A new operator have to start with calling it's precondition. The precondition must
get the same parameters as the operator itself. If the precondition is false, than the operator is by no
means executable. If it's true, than the state transitions must be executed. Than we have to check if
we are out of the state-space with the IsState predicate. If this is true, so we've remained inside the
state-space, then we are ready, the operator can be executed. Else it can't be executed and the state
transition must be withdrawn too.
8.2.1 SOURCE CODE
/// <summary>
/// The BlindState is only here for demonstration.
/// It shows how to write the operators and connect them to the super operator.
/// </summary>
abstract class BlindState : AbstractState
{
// Here we need to give the fields that contain the inner state.
// The operators execute an inner state transition.
// First the basic operators must be written.
// Every operator has a precondition.
// Every operator's postcondition is the same as it's SateE predicate.
// The operator returns a true, if it's executable and a false if it is not executable.
// An operator is executable if it's true for the inner state,
// for the preconditions and for the postcondition after state transition.
// This is the first basic operator.
private bool op1()
{
// If the precondition is false, the operator is not executable.
if (!preOp1()) return false;
// state transition
//
// TODO: We need to complete the code here!
//
// Checking the postcondition, if it is true, the operator is executable.
if (IsState()) return true;
// Else the inner state transition must be withdrawn,
//
// TODO: We need to complete the code here!
//
// and have to return that the operator is not executable.
return false;
}
// The precondition of the first basic operator. The name of the precondition is usually: pre+name of the operator.
// It helps understanding the code, but we can calmly differ from it.
private bool preOp1()
{
// If the precondition is true, it returns as true.
return true;
}
// Another operator.
private bool op2()
{
if (!preOp2()) return false;
// State transation:
// TODO: We need to complete the code here!
if (IsState()) return true;
// Else the inner state transition must be withdrawn
// TODO: We need to complete the code here!
return false;
}
private bool preOp2()
{
// If the precondition is true, it returns as true.
return true;
}
// Let's see what is the situation if the operator has parameters.
// This time one operator equals more basic operators.
private bool op3(int i)
{
// The precondition must be invited with the same parameters as the operator itself.
if (!preOp3(i)) return false;
// State transition:
// TODO: We need to complete the code here!
if (IsState()) return true;
// Else the inner state transition must be withdrawn
// TODO: We need to complete the code here!
return false;
}
// The precondition's parameter list is the same as the operator's parameter list.
private bool preOp3(int i)
{
// If the precondition is true than it return as true. The precondition depends on the parameters.
return true;
}
// This is the super operator. The basic operators can be called through this one.
// With parameter i, we say which operator we want to invite.
// Usually we invite it from a "for" cycle, which runs from 0 to NumberOfOPs.
public override bool SuperOperator(int i)
{
switch (i)
{
// We need to list all of the basic operators here.
// We have to insert a new operator here.
case 0: return op1();
case 1: return op2();
// The operators with parameters equal to more basic operators, so we have more lines for them here.
// The number of lines depends on the task.
case 3: return op3(0);
case 4: return op3(1);
case 5: return op3(2);
default: return false;
}
}
// Returns the number of basic operators.
public override int NumberOfOps()
{
// we have to return the number of the last case.
// If we extend the number of operators, this number must be extended too.
return 5;
}
}
8.3 A STATE CLASS EXAMPLE: HUNGRYCAVALRYSTATE
In this chapter we can see a well written state-space representation. We can see that every State
class must originate from the AbstractState class. We can see that how the different methods should
be realized. As to solve one's own task, only a personal State class must be written, it worth starting
from this example or the one in the next chapter.
8.3.1 SOURCE CODE
/// <summary>
/// This is the state-space representation of the "hungry cavalry" problem.
/// The cavalry must get from the place of the station, the upper left corner,
/// to the cantina, which is in the bottom right corner.
/// We represent the table with a (N+4)*(N+4) matrix.
/// The outer 2 lines are margins, the inside part is the table.
/// With the use of the margins, it much more easy to write the IsState predicate.
/// The meaning of 0 is empty. The meaning of 1 is: here is the horse.
/// With a 3*3 table, the initial state is the following:
/// 0,0,0,0,0,0,0
/// 0,0,0,0,0,0,0
/// 0,0,1,0,0,0,0
/// 0,0,0,0,0,0,0
/// 0,0,0,0,0,0,0
/// 0,0,0,0,0,0,0
/// 0,0,0,0,0,0,0
/// It is visible from the above representation that it's enough to store the position of the horse,
/// as it is the horse only that is on the table. So the initial state is (beginning from the upper left corner):
/// x = 2
/// y = 2
/// The goal state (I'm going to the bottom right corner):
/// x = N+1
/// y = N+1
/// Operators:
/// The 8 possible horse steps.
/// </summary>
class HungryCavalryState : AbstractState
{
// By default it runs on a 3*3 chess board.
static int N = 3;
// The fields describing the inner state.
int x, y;
// Sets the inner state for the initial state.
public HungryCavalryState()
{
x = 2; // We start from the upper left corner, which on the (2,2) coordinate
y = 2; // due to the margin.
}
// Sets the inner state for the initial state.
// We can also set the size of the board here.
public HungryCavalryState(int n)
{
x = 2;
y = 2;
N = n;
}
public override bool IsGoalState()
{
// The bottom right corner is on (N+1,N+1) due to the margin.
return x == N + 1 && y == N + 1;
}
public override bool IsState()
{
// the horse is not on the margin
return x >= 2 && y >= 2 && x <= N + 1 && y <= N + 1;
}
private bool preHorseStep(int x, int y)
{
// if it's a correct horse step, if not then return false
if (!(x * y == 2 || x * y == -2)) return false;
return true;
}
/* This is my operator, returns true if it's executable,
* else it returns false.
* Parameters:
* x: x direction move
* y: y direction move
* The precondition checks if the move is a horse step.
* The postcondition checks if we remained on the board.
* Example:
* HorseStep(1,-2) meaning: 2 up, 1 right.
*/
private bool HorseStep(int x, int y)
{
if (!preHorseStep(x, y)) return false;
// If the precondition is true, we execute the
// state transation.
this.x += x;
this.y += y;
// The postcondition is always equals to the IsState.
if (IsState()) return true;
// If the postcondition is not true, than it must be withdrawn.
this.x -= x;
this.y -= y;
return false;
}
public override bool SuperOperator(int i)
{
switch (i)
{ // We list the 8 possible horse steps here
case 0: return HorseStep(1, 2);
case 1: return HorseStep(1, -2);
case 2: return HorseStep(-1, 2);
case 3: return HorseStep(-1, -2);
case 4: return HorseStep(2, 1);
case 5: return HorseStep(2, -1);
case 6: return HorseStep(-2, 1);
case 7: return HorseStep(-2, -1);
default: return false;
}
}
public override int NumberOfOps() { return 8; }
// when writing it, we substract the width of the margin from x and y.
public override string ToString() { return (x-2) + " : " + (y-2); }
public override bool Equals(Object a)
{
HungryCavalryState aa = (HungryCavalryState)a;
return aa.x == x && aa.y == y;
}
// If two példánys are equal, than their hasítókóds are equal too.
public override int GetHashCode()
{
return x.GetHashCode() + y.GetHashCode();
}
}
8.4 ANOTHER STATE CLASS EXAMPLE
In this chapter we can see the state-space representation of the well-known 3 monks 3 cannibals
problem. As all State classes, this has to originate from the AbstractState class too. As to solve one's
own task, only a personal State class must be written, it worth starting from this example or the one
in the previous chapter.
8.4.1 THE EXAMPLE SOURCE CODE OF THE 3 MONKS AND 3 CANNIBALS
/// <summary>
/// The state-space representation of the "3 monks, 3 cannibals" problem.
/// Or rather it's generalization to any number of monks and cannibals.
/// Problem: there are 3 monks and 3 cannibals on the left bank of the river.
/// All of them must be transported to the other side of the river.
/// For this, a 2-man boat is available.
/// One man is enough to get to the other side, more than two people can't fit in.
/// If someone goes to the other side, he must get out, can't stay in the boat.
/// The problem is, if there are more cannibals than monks on either side of the river,
/// then the cannibals eat the monks.
/// Initial state:
/// 3 monks on the left side.
/// 3 cannibals on the left side.
/// The boat is on the left side.
/// 0 monks on the right side.
/// 0 cannibals on the right side.
/// This state is described with the following organized quintet:
/// (3,3,'L',0,0)
/// The goal state:
/// (0,0,'R',3,3)
/// Operator:
/// Op(int m, int c):
/// m monks go to the other side and
/// c cannibals go to the other side.
/// Possible parameters:
/// Op(1,0): 1 monks goes to the other.
/// Op(2,0): 2 monks goes to the other side.
/// Op(1,1): 1 monk and 1 cannibal goes to the other side.
/// Op(0,1): 1 cannibal goes to the other side.
/// Op(0,2): 2 cannibal goes to the other side.
/// </summary>
class MonksAndCannibalsState : AbstractState
{
int m; // the number of monks all together
int c; // the number of cannibals all together
int ml; // the number of monks on the left side
int cl; // the number of cannibals on the left side
char b; // Where the boat is? It's value is either 'L' or 'R'.
int mr; // the number of monks on the right side
int cr; // the number of cannibals on the right side
public MonksAndCannibalsState(int m, int c) // sets the initial state
{
this.m = m;
this.c = c;
ml = m;
cl = c;
b = 'L';
mr = 0;
cr = 0;
}
public override bool IsState()
{ // true, if the monks are safe
return ((ml >= cl) || (ml == 0)) &&
((mr >= cr) || (mr == 0));
}
public override bool IsGoalState()
{
// true, if everyone got to the other side
return mr == m && cr == c;
}
private bool preOp(int m, int c)
{
// The boat can carry 2 people, at least 1 is needed for paddling.
// At the end, it's needles to check it as the super operator invites it according to this.
if (m + c > 2 || m + c < 0 || m < 0 || c < 0) return false;
// m monks and c cannibals are only transportable
// if the boat's side has at least that many.
if (b == 'L')
return ml >= m && cl >= c;
else
return mr >= m && cr >= c;
}
// Transports m monks and c cannibals to the other side.
private bool op(int m, int c)
{
if (!preOp(m, c)) return false;
MonksAndCannibalsState mentes = (MonksAndCannibalsState)Clone();
if (b == 'L')
{
ml -= m;
cl -= c;
b = 'R';
mr += m;
cr += c;
}
else
{
ml += m;
cl += c;
b = 'L';
mr -= m;
cr -= c;
}
if (IsState()) return true;
ml = without.ml;
cl = without.cl;
b = without.b;
mr = without.mr;
cr = without.cr;
return false;
}
public override int NumberOfOps() { return 5; }
public override bool SuperOperator(int i)
{
switch (i)
{
case 0: return op(0, 1);
case 1: return op(0, 2);
case 2: return op(1, 1);
case 3: return op(1, 0);
case 4: return op(2, 0);
default: return false;
}
}
public override string ToString()
{
return ml + "," + cl + "," + b + "," + mr + "," + cr;
}
public override bool Equals(Object a)
{
MonksAndCannibalsState aa = (MonksAndCannibalsState)a;
// mr and cr are countable, so no need for checking
return aa.ml == ml && aa.cl == cl && aa.b == b;
}
// If two instances are equal, than their hash codes are equal too.
public override int GetHashCode()
{
return ml.GetHashCode() + cl.GetHashCode() + b.GetHashCode();
}
}
8.5 THE VERTEX CLASS
In this chapter, we get to know the vertex class. Fortunately, we don't need to change this if we want
to solve one's one task with backtrack or with depth-first method. If we want to implement some
other algorithm, like the uniform-cost method, instead of an algorithm based on depth, then we need
to modify it so the state will contain the the value of the cost function.
In this version, the vertex contains a state, it's depth and it's parent. It supports extending and with
the help of the super operators, we can use the operators one by one.
As the vertex knows it's parent, for recording the path it's enough to store the last vertex of the path.
This means that for the solution, it's enough to return the terminal vertex where the solution leads
too.
8.5.1 SOURCE CODE
/// <summary>
/// The vertex contains a state, the vertex's depth and the vertex's parent.
/// So the vertex represents a whole path till the starting vertex.
/// </summary>
class Vertex
{
// The vertex contains a state, it's depth and it's parent
AbstractState state;
int depths;
Vertex parent; // Going upwards on the parents we get to the starting vertex.
// Constructor:
// Sets the inner state on the starting vertex.
// It's the inviter's responsibility to invite it with the initial state.
// The depth of the starting vertex is 0 and it has no parent.
public Vertex(AbstractState initialState)
{
state = initialState;
depth = 0;
parent = null;
}
// A new child creates a vertex.
// We need to invite an applicable operator, only after that it is ready.
public Vertex(Vertex parent)
{
state = (AbstractState)parent.state.Clone();
depth = parent.depth + 1;
this.parent = parent;
}
public Vertex GetParent() { return parent; }
public int GetDepth() { return depth; }
public bool TerminalVertexE() { return state.IsGoalState(); }
public int NumberOfOps() { return state.NumberOfOps(); }
public bool SuperOperator(int i) { return state.SuperOperator(i); }
public override bool Equals(Object obj)
{
Vertex cs = (Vertex)obj;
return state.Equals(cs.state);
}
public override int GetHashCode() { return state.GetHashCode(); }
public override String ToString() { return state.ToString(); }
// Executes all the executable operators.
// Returns the so created new vertices.
public List<Vertex> Extending()
{
List<Vertex> newVertices = new List<Vertex>();
for (int i = 0; i < NumberOfOps(); i++)
{
// We create a new child vertex.
Vertex newVertex = new Vertex(this);
// We try the i
th
basic operator? Is it executable?
if (newVertex.SuperOperator(i))
{
// If yes, we add it to the new ones.
newVertices.Add(newVertex);
}
}
return newVertices;
}
}
8.6 THE GRAPHSEARCH CLASS
In this chapter we introduce the common concepts of the graph searcher algorithms. This class
probably won't need modification, not even in the case when a new graph searcher algorithm has
been written. Al graph searches must originate from this class.
It has one abstract method, the Search. This is what needs to be realized in children classes. In case
of the backtrack, it will be the backtrack itself, in case of the depth-first method, it will be the
depths search. The Search returns with a vertex. This must be the terminal vertex, if a solution
exists, in other case, it must be null. The returning null value indicates that there is no solution.
The solution can be written with the WriteSolution method. This is only a small aid, its use is not
required. If we decide on using it, there is an example in the main program for its call.
Every graph searcher starts searching in the starting vertex, so we prescribe a constructor too. This
must be invited by every children class.
8.6.1 SOURCE CODE
/// <summary>
/// The ancestor of all graph searchers.
/// The graph searchers only need to realize the Search method.
/// This will return a terminal vertex, if there is a solution or a null if there isn't.
/// From the terminal vertex, the solution can be made by going up on the parent references.
/// </summary>
abstract class GraphSearch
{
private Vertex startingVertex; // The starting vertex vertex.
// Every graphsearxher starts searching in the starting vertex.
public GraphSearch(Vertex startingVertex)
{
this. startingVertex = startingVertex;
}
// It's better if the starting vertex is private, but the children classes can request it.
protected Vertex GetStartingVertex () { return startingVertex; }
/// If there is a solution, namely there is a path in the state-space graph,
/// that leads from the starting vertex to the terminal vertex,
/// it returns a solution, else it's null.
/// The solution is given as a terminal vertex.
/// From this terminal vertex, the solution can be made by going up on the parent references in reverse order.
public abstract Vertex Search();
/// <summary>
/// Writes the solution based on a terminal vertex.
/// It assumes that by going up on the references of the parent vertex we will get to the starting vertex.
/// It reverses the order of the vertices to write the solution correctly.
/// If a vertex is null, it writes that there is no solution.
/// </summary>
/// <param name="oneTerminalVertex">
/// The terminal vertex representing the solution or null.
/// </param>
public void writingSolution(Vertex oneTerminalVertex)
{
if ( oneTerminalVertex == null)
{
Console.WriteLine("No solution");
return;
}
// The order of the vertices must be reversed.
Stack<Vertex> solution = new Stack<Vertex>();
Vertex curVertex = oneTerminalVertex;
while (curVertex != null)
{
solution.Push(curVertex);
curVertex = curVertex.GetParent();
}
// It has been reversed, can be written.
foreach(Vertex cur in solution) Console.WriteLine(cur);
}
}
8.7 THE BACKTRACK CLASS
In this chapter we introduce a recursive solution of the backtrack graph search. Different
constructors can be used to create a backtrack with depth limit, with memory or the combination of
these.
It worth trying to rewrite the below realization to a loop based solution. It is a great practice, and if
it doesn't work, the original is still there to create the hand-in program. In this, we don't need to even
touch this code. It is a well-tested code.
8.7.1 SOURCE CODE
/// <summary>
/// Class that realizes the the backtrack graph searcher algorithm.
/// It contains the three basic backtrack algorithms in one. These are:
/// - the basic backtrack
/// - the backtrack with depth limit
/// - the backtrack with memory
/// The backtrack with branch-limit hasn't been worked out here.
/// </summary>
class BackTrack : GraphSearch
{
int limit; // If it's not null, than the searcher has depth limit.
bool memory; //if it's true, the searcher has memory.
public BackTrack(Vertex startingVertex, int limit, bool memory) : base(startingVertex)
{
this.limit = limit;
this.memory = memory;
}
// there is neither a depth limit, nor a memory
public BackTrack(Vertex startingVertex) : this(startingVertex, 0, false) { }
// searcher with depth limit
public BackTrack(Vertex startingVertex, int limit) : this(startingVertex, limit, false) { }
// searcher with memory
public BackTrack(Vertex startingVertex, bool memory) : this(startingVertex, 0, memory) { }
// The search starts from the starting vertex.
// Returns a terminal vertex. We can get to this terminal vertex from the starting vertex.
// If there is no such vertex, it returns a null value.
public override Vertex Search() { return Search(GetStartingVertex()); }
// The recursive realization of the searcher algorithm.
// As it is recursive, the backstep is the "return null".
private Vertex Search(Vertex curVertex)
{
int depth = curVertex.GetDepth();
// checking the depth limit
if (limit > 0 && depth >= limit) return null;
// the use of memory to remove circles
Vertex curParent = null;
if (memory) curParent = curVertex.GetParent();
while (curParent != null)
{
// We check if we have been in this state previously. If yes, then backstep.
if (curVertex.Equals(curParent)) return null;
// Going back on the parent trail.
curParent = curParent.GetParent();
}
if (curVertex.TerminalVertexE())
{
// We have the solution, the terminal vertex must be returned.
return curVertex;
}
// We invite the basic operators here through the super operator.
// If any of them can be applied, we create a new vertex,
// and invite myself recursively.
for (int i = 0; i < curVertex.NumberOfOperators(); i++)
{
// We create the new child vertex.
// This will be done only if we apply an applicable operator on it.
Vertex newVertex = new Vertex(curVertex);
// We try the i
th
basic operator. Is it applicable?
if (newVertex.SuperOperator(i))
{
// If yes, then we invite myself recursively to the new vertex.
// If it returns with a value other than null, then a solution has been found.
// If it returns with a null value, we have to try the next basic operator.
Vertex terminal = Search(newVertex);
if (terminal != null)
{
// We return the terminal vertex that represents the solution.
return terminal;
}
// The operator should be withdrawn on the else-branch.
// This is required if there was no cloning in the creation of the new child.
// As we cloned, this part is empty.
}
}
// If we tried all the operators and none of them has given a solution then backstep.
// After the backstep, we try to use the next basic operator one level higher.
return null;
}
}
8.8 THE DEPTHFIRSTMETHOD CLASS
In this chapter, we introduce the depth-first search in two versions. The first version uses loop-
checking, as the original depth searcher uses it too. The other version doesn't use it, so it is much
faster. The second version should only be used if there are no loops in the state-space graph, else the
searcher may enter an infinite cycle.
Both versions assume that the starting vertex is not terminal. If this cannot be assumed, than we
need to test the vertex that we selected for expanding if it is a terminal vertex or not. Otherwise, it's
enough to test the new vertices, as it is done in the code below.
It is a well-known fact that the depth-first search should be done with a stack, while the breadth-first
search should be done with a queue. According to this, we store the open vertices in a stack. Due to
this, on the top of the stack, it will be the open vertex with the greatest depth. This greatly hastens
the search, as we don't need to look for the vertex with the greatest depth.
We advice that the reader would write the breadth-first search based on the code below. This
shouldn't cause any trouble. It is a bit more complicated task to write the uniform-cost method
based on these information. To achieve this, the Vertex class must be completed with the cost, and
instead of the stack, a sorted list should be used to find the open vertex with the lowest cost. With
the uniform-cost method, we also need to pay attention to the fact that more paths may lead to one
vertex, but we only need to store the one with the lowest cost. The A-algorithm, where there is a
heuristic too, is even more exciting.
8.8.1 SOURCE CODE
/// <summary>
/// A graph search class that realizes the depth search algorithm.
/// This realization of the depth search assumes that the starting vertex is not terminal.
/// The open vertices are stored in a stack.
/// </summary>
class DepthFirstMethod : GraphSearch
{
// In the depth search, it worth storing the open vertices in a stack,
// as in this way, the vertex with the greatest depth will be on the top of the stack.
// So we don't need to search for the open vertex with the greatest depth.
Stack<Vertex> Open; // Set of the open vertices.
List<Vertex> Closed; // Set of the closed vertices.
bool circleChecking; // If it is false, it may enter into an infinite cycle.
public DepthFirstMethod(Vertex startVertex, bool circleChecking) :
base (startingVertex)
{
Open = new Stack<Vertex>();
Open.Push(startingVertex); // at the beginning, only the starting vertex is open
Closed = new List<Vertex>(); // at the beginning, the set of closed vertices is empty
this.circleChecking = circleChecking;
}
// The default value of circle checking is true.
public DepthFirstMethod(Vertex startingVertex) : this(startingVertex, true) { }
// The is the point where the search for the solution starts.
public override Vertex Search()
{
// If we don't need the circle checking, the algorithm will be much faster.
if (circleChecking) return SearchingTerminalVertex();
return SearchingTerminalVertexFast();
}
private Vertex SearchingTerminalVertex()
{
// Till the set of open vertices is not empty.
while (Open.Count != 0)
{
// This is the open vertex with the greatest depth.
Vertex C = Open.Pop();
// We expand this one.
List<Vertex> newVertices = C.Expending();
foreach (Vertex D in newVertices)
{
// I'm ready if we have found the terminal vertex.
if (D.TerminalVertexE()) return D;
// We only pick up the vertices to the list of open vertices
// that haven't been included neither in the set of open, nor in the set of closed vertices.
// The Contains invites the Equals method that was written in the Vertex class.
if (!Closed.Contains(D) && !Open.Contains(D)) Open.Push(D);
}
// We made the expanded vertex to be a closed vertex.
Closed.Add(C);
}
return null;
}
// This should only be used if we are sure that the state-space graph does not contain a circle!
// Else it will probably cause an infinite cycle.
private Vertex SearchingTerminalVertexFast()
{
while (Open.Count != 0)
{
Vertex C = Open.Pop();
List<Vertex> newVertices = C.Expanding();
foreach (Vertex D in newVertices)
{
if (D.TerminalVertexE()) return D;
// If there is no circle, it is needless to check if D had been among the open or closed vertices.
Open.Push(D);
}
// if there is no circle, it is needless to made C closed.
}
return null;
}
}
8.9 THE MAIN PROGRAM
In this chapter we can see how the different parts should be merged. It's very important for the
starting vertex to get the initial state. We have to give the starting vertex to the constructor of the
graph searcher. These are all the responsibilities of the main program.
It worth checking that we can invite different searchers on the same starting vertex and they will
end in slightly different results. Another worthy thing is playing with the constants. Note that a
depth limit of 10 is enough for the example of Hungry Cavalryman, but it is not enough for the 3
monks, 3 cannibals. It's also interesting to check that what size of a task will make the problem
difficult to solve. It may be salient to, that the depth-first method may run out of memory with
greater problems, where the backtrack has plenty of memory. It also worth noting that reordering
the sequence of operators in the super operator influences the run time greatly. A brave man may
rewrite the super operator to use heuristics.
It is visible that it doesn't matter which graph searcher algorithm we use, the Searching method
must be invited. The writeSolution method should be invited on the vertex that was returned by that
method.
8.9.1 SOURCE CODE
class Program
{
static void Main(string[] args)
{
Vertex startingVertex;
GraphSearch search;
Console.WriteLine("We solve the Hungry Cavalryman problem on a 4x4 board.");
startingVertex = new Vertex(new HungryCavalryState(4));
Console.WriteLine("The searcher is a backtrack searcher with memory and depth limit of 10.");
search = new BackTrack(startingVertex, 10, true);
search.writeSolution(search.Searching());
Console.WriteLine("The searcher is a depth searcher with circle checking.");
search = new DepthFirstMethod(startingVertex, true);
search.writeSolution(search.Searching());
Console.WriteLine("We solve the 3 monks, 3 cannibals problem.");
startingVertex = new Vertex(new MonksAndCannibalsState(3,3));
Console.WriteLine("The searcher is a backtrack searcher with memory and depth limit of 15.");
search = new BackTrack(startingVertex, 15, true);
search.writeSolution(search.Searching());
Console.WriteLine("The searcher is a depth searcher with circle checking.");
search = new DepthFirstMethod(startingVertex, true);
search.writeSolution(search.Searching());
Console.ReadLine();
}
}
BIBLIOGRAPHY
(1) Stuart J. Russell, Peter Norvig: Mesterséges intelligencia modern megközelítésben
Panem – Prentice Hall, 2000 (Hungarian title)
(2) Pásztorné Varga Katalin, Várterész Magda: A matematikai logika
alkalmazásszemléletű tárgyalása
Panem, 2003
(3) Dr. Várterész Magda: Mesterséges intelligencia
http://www.inf.unideb.hu/~varteres/mi/tartalom.htm
(4) Dr. Kovásznai Gergely: Logikai programozás
http://aries.ektf.hu/~kovasz/logprog/logikai_programozas.pdf
(5) CLIPS
http://www.ghg.net/clips/CLIPS.html
(6) SWI-Prolog
http://www.swi-prolog.org

Table of Contents
1 Introduction...................................................................................................................................... 4 2 The History of Artificial Intelligence............................................................................................... 7 2.1 Early Enthusiasm, Great Expectations (Till the end of the 1960s)...........................................7 2.2 Disillusionment and the knowledge-based systems (till the end of the 1980s)........................ 8 2.3 AI becomes industry (since 1980).............................................................................................9 3 Problem Representation..................................................................................................................10 3.1 State-space representation.......................................................................................................10 3.2 State-space graph.................................................................................................................... 11 3.3 Examples.................................................................................................................................12 3.3.1 3 jugs............................................................................................................................... 12 3.3.2 Towers of Hanoi.............................................................................................................. 15 3.3.3 8 queens...........................................................................................................................17 4 Problem-solving methods...............................................................................................................20 4.1 Non-modifiable problem-solving methods.............................................................................22 4.1.1 The Trial and Error method.............................................................................................23 4.1.2 The trial and error method with restart............................................................................23 4.1.3 The hill climbing method................................................................................................ 24 4.1.4 Hill climbing method with restart................................................................................... 25 4.2 Backtrack search..................................................................................................................... 25 4.2.1 Basic backtrack............................................................................................................... 26 4.2.2 Backtrack with depth limit.............................................................................................. 29 4.2.3 Backtrack with cycle detection....................................................................................... 31 4.2.4 The Branch and Bound algorithm................................................................................... 33 4.3 Tree search methods................................................................................................................34 4.3.1 General tree search..........................................................................................................35 4.3.2 Systematic tree search..................................................................................................... 37 4.3.2.1 Breadth-first search................................................................................................. 37 4.3.2.2 Depth-first search.................................................................................................... 38 4.3.2.3 Uniform-cost search................................................................................................ 40 4.3.3 Heuristic tree search........................................................................................................41 4.3.3.1 Best-first search.......................................................................................................42 4.3.3.2 The A algorithm.......................................................................................................43 4.3.3.3 The A* algorithm.....................................................................................................47 4.3.3.4 The monotone A algorithm......................................................................................49 4.3.3.5 The connection among the different variants of the A algorithm............................ 51 5 2-player games................................................................................................................................52 5.1 State-space representation.......................................................................................................53 5.2 Examples.................................................................................................................................53 5.2.1 Nim..................................................................................................................................53 5.2.2 Tic-tac-toe....................................................................................................................... 54 5.3 Game tree and strategy............................................................................................................56 5.3.1 Winning strategy............................................................................................................. 58 5.4 The Minimax algorithm.......................................................................................................... 58 5.5 The Negamax algorithm..........................................................................................................61 5.6 The Alpha-beta pruning.......................................................................................................... 63 6 Using artificial intelligence in education........................................................................................66 6.1 The problem............................................................................................................................ 66 6.1.1 Non-modifiable searchers............................................................................................... 67

6.1.2 Backtrack searchers.........................................................................................................69 6.1.3 Tree Search Methods.......................................................................................................70 6.1.4 Depth-First Method.........................................................................................................72 6.1.5 2-Player game programs..................................................................................................73 6.2 Advantages and disadvantages................................................................................................74 7 Summary.........................................................................................................................................75 8 Example programs..........................................................................................................................77 8.1 The AbstractState class........................................................................................................... 77 8.1.1 Source code..................................................................................................................... 77 8.2 How to create my own operators?.......................................................................................... 78 8.2.1 Source code..................................................................................................................... 78 8.3 A State class example: HungryCavalryState...........................................................................80 8.3.1 Source code..................................................................................................................... 80 8.4 Another State class example................................................................................................... 82 8.4.1 The example source code of the 3 monks and 3 cannibals............................................. 82 8.5 The Vertex class...................................................................................................................... 84 8.5.1 Source code..................................................................................................................... 85 8.6 The GraphSearch class............................................................................................................86 8.6.1 Source code..................................................................................................................... 86 8.7 The backtrack class................................................................................................................. 87 8.7.1 Source code..................................................................................................................... 87 8.8 The DepthFirstMethod class................................................................................................... 88 8.8.1 Source code..................................................................................................................... 89 8.9 The Main Program.................................................................................................................. 90 8.9.1 Source code..................................................................................................................... 90 Bibliography..................................................................................................................................... 92

we solve the real problem. which are relatively small compared to the capabilities of a computer. We will bring out such „clever” algorithms. informatics helps to plant the model's solution into reality. etc. So our computer remains as thick as two short planks. the engineering sciences. If the well-know 8 Queens Problem should be played with 1-ton iron queens. This is the task that should be started developing in high school. we can be sure that the solution is right. but we exploit the no more than two good qualities that a computer has. With the help of the solution found in the model. At the first step. We solve the modelled problem. the answer from a mathematically educated colleague comes in an instant: It depends on what the definition is? If artificial intelligence is when the computer beats us in chess. The second step uses an abstract idea system. which is required to think over and execute the algorithms published in this note. then we are again on the right track to execute artificial intelligence. where mathematics and logic helps to work on the abstract objects. This lecture note uses artificial intelligence in the first sense. we would also need a massive hoisting crane. This is where the main point of our note comes in: the human creativity required by the artificial intelligence. This requires the expansion of the following skills: Model creation by the abstraction of the reality System approach It would be worthwhile to add algorithmic thinking to the list above. All steps are helped by different branches of science. Alas. and the searching would take a few days and a few hundreds of diesel oil till . subtraction. so quickly and correctly applying the steps dictated by the graph search algorithms will result in a fast-solved Cube and due to the correctness. This is all nice. However. At the same time. that can be used to solve the so called graph searching problems. if our expectation is that the computer should understand what we say. chemistry. To represent a problem in a way that it's graph would keep small. then we are far away from it. The problems that can be rewritten into a graph search – such as chess – can be solved by the computer. the help comes from the sciences that describe reality: physics. etc. which are: The computer can do algebraic operations (addition. The solution of a problem is the following in the case of applying artificial intelligence: We model the real problem. then we are very close to attain artificial intelligence. Searching can be quite difficult and expensive in reality. We will talk about this in a subsequent chapter.) very fast. the computer will not become clever in the ordinary meaning of the word if we implement these algorithms. but why can't we solve the existing problem in reality at once? Why do we need modelling? The answer is simple. we can easily find a problem that's graph representation is so huge.1 INTRODUCTION Surely everyone have thought about what artificial intelligence is? In most cases. it will be able to systematically examine a graph in search of a solution. So we exploit the fact that such problems that are to difficult for a human to see through – like the solution of the Rubik Cube – are represented in graphs. It does these correctly. that even the fastest computers are unable to quickly find a solution in the enormous graph. If the definition is to drive a land rover through a desert from point A to point B. At last. at best.

That is why we need modelling. but it may not if we chop down a tree unless the number of the trees is not an important detail in the solution of the problem. Algorithms. We have a separate chapter on this topic. a house can be ruined at this point. So we can take the definition of the operators to be the next step after choosing the framework. We need to identify the possible „operators” that can be used to change reality. it's an operator. that we either took transformational. Unfortunately. is the most important in the artificial intelligence's aspect. let's see the different steps in detail. It is easier and cheaper to search for a solution in an abstract space. the possible states and the operators (including the pre and post condition of the operators). We need to go through the following steps to solve the modelled problem: Chose a framework that can solve the problem. which rarely contains unnecessary informations. appear in secondary school? Fortunately. This doesn't mean that we have to implement this algorithm. finding the important parts in a text. . like the depth of the wall. so the solution's error content can also be given. The operator is a thing. We have to count and measure the important parts. Unfortunately. We only need to implement. We will see that our model. non-circle graph. We are dealing with this question in connection with the „will-thehouse-collapse” . also know as state space can be given with the initial state. the Prolog interpreter uses backtrack search. For example. The framework solves the problem. It is also important to know that measuring reality is always disturbed with errors. because if we neglect an important issue. How does this problem.issue. Modelling the existing problem: We magnify the parts of the problem that are important for the solution and neglect the ones that are not. the possible groupings are: Algorithms. The frameworks may differ from each other in many ways. Choosing the framework that is able to solve our model means choosing the algorithm that can solve the modelled problem. The writer of the exercise usually takes it the other way round and we need to find some additional information which is hidden in the text. the rules that describe the model in Prolog. that surly find the solution in a limited graph. that changes the part of reality that is important for us. the identification of the „operators”. that surly find the solution in a limited. For the answer. the set of end states. the addition of the the initial errors can be given. namely. this step is influenced by the fact. Modelling the existing problem is called state space representation in artificial intelligence. it's usually a maths exercise. With the tools of Numeric mathematics. Regarding artificial intelligence. Algorithms.we find a solution. when we move in chess. Set the model in the framework. that give an optimal solution according to some point of view.(that creates a state from another state) or problem reduction (that creates more states from another state) operators in the state space representation. What guarantees that the solution found in the abstract space will work in reality? So. The third step. the house may collapse. what guarantees that a house built this way will not collapse? This is a difficult question. which is the second step. it takes from one well-describable state into another.

we need to know what do we mean under 'solution'. that is a correct solution inside the model. Here. The warranty for this is the fact that the algorithms introduced in the notes are correct. we can trust our solution in the same extent as we can trust in logics. which is guaranteed by the correctness of the algorithms and the fact that logic is based on reality! . then the solution is a sequence of steps about how to build the house.and postconditions). Now. If we don't trust in the solution given by the model. our last task is to implement the model in the framework. and the framework will solve the problem if it is able to do it. which is replanting the abstract model into reality. Of course. We have no other task than executing the steps of the model's solution in reality. We only need to push the button. if the initial state is that we have enough material to build a house and the end state is that a house had been built according to the design. If we haven't messed up neither of the steps. First of all. Solution is a sequence of steps (operator applications). There is only one question left: will the house collapse? The answer is definitely 'NO'. This is usually means setting the initial state. for example). the end condition and the operators (with pre. If we found that the step is impossible.If we have the adequate framework. then the house will stand. which was creating the model. So. that leads from the initial state into an end state. we can face that a step. assume that we have got a solution. that was quite simple in the model (like move the queen to the A1 field) is difficult if not impossible in reality. we can mess up the implementation of the model (by giving an incorrect end condition. if we haven't done any mistake at the previous step. than our model is incorrect. but if we manage to evade this tumbler. namely by logical methods it can be proven that if they result in a solution. The last step is to solve the real problem with the solution that we found in the model. then it worth trying it in small. and will not do at the next step.

the researchers drew up ambitious plans (world champion chess software. Formally it was created in 1956. GREAT EXPECTATIONS (TILL OF THE 1960S) THE END In a way. universal translator machine) and the main direction of research was to write up general problem solving methods. learn. the early years of AI were full of successes. probability. along with the view that the mind is created by the functioning of some physical system. although some electronic mind of the world” :) researches had already been going on for 5 years. In this era. computers were only though to be capable of doing arithmetical tasks. and manipulate a world that is much larger and more complex than itself? And what if we would like to construct something with such capabilities? AI is one of the newest field of science. Philosopher have been trying to understand for more than 2000 years what mechanism we use to sense. „Faster than Einstein” . when its Figure 1. If we consider the primitive computers and programming tools of that age. decisionmaking. 2. and think. AI's history can be broken down into three major periods. Many thought that these „electrical masterminds” have infinite potencies regarding executing intelligence. understand. modelling intelligent thinking and behaving with computers proved much more difficult than many have thought at the beginning. The Artificial Intelligence (AI) deals with the ultimate challenge: How can a (either biological or electronic) mind sense.2 THE HISTORY OF ARTIFICIAL INTELLIGENCE Studying the intelligence is one of the most ancient scientific discipline. In the meantime.became a typical newspaper article. foretell.1 EARLY ENTHUSIASM. The early optimism of the 1950s: „The smallest name was created. it was astonishing to think that the computer is – even if far from it – capable of doing clever things. these philosophical theories made the formal theory of logic. The scientific analysis of skills in connection with intelligence was turned into real theory and practice with the appearance of computers in the 1950s. From the 2000 years old philosophical tradition the theory of reasoning and learning have developed. remember. that even a few years before. Among others. Allen Newell and Herbert Simon created a general problem solving application (General . and calculation develop from mathematics. and the fact.

Richard Karp. Robinson. it was thought that simple syntactic transformations based on the English and Russian grammar and word substitution will be enough to define the precise meaning of a sentence. The other difficulty was that many problems that were tried to solve by the AI were untreatable. it gave the following text: „The vodka is strong. According to the anecdote. 1972). The early softwares were usable because the worlds they handled contained only a few objects. In the early era. and the fact that general knowledge about a topic is necessary to resolve the ambiguities. when the famous „The spirit is willing. At the beginning of the 1970s. reached the effectiveness of human experts. Arthur Samuel wrote an application that played Draughts and whose game power level reached the level of the competitors. Prolog. The early AI softwares were trying step sequences based on the basic facts about the problem that should be solved. which outgrew into the primary language of AI programming. In 1958. From the end of the 1960s.2 DISILLUSIONMENT AND THE KNOWLEDGE-BASED SYSTEMS (TILL THE END OF THE 1980S) The general-purpose softwares of the early period of AI were only able to solve simple tasks effectively and failed miserably when they should be used in a wider range or on more difficult tasks. There is a typical story in connection with the early computer translations. developing the so-called expert systems were emphasised. Prolog is a remarkably . In computational complexity theory. and put up a significantly better show than novice physicians. but it became a strong opponent after playing a few days with itself. as his application quickly learnt to play better than Samuel himself.Program Solver. and on the field of heuristic search and methods for handling uncertainty. At the beginning. 1971. serious accomplishments were born in the theory of resolution theorem proving (J. This was confuted in theory by the results in connection with NP-completeness. After the Sputnik's launch in 1957. One of these was Herbert Gelernter's Geometry Theorem Prover. 2. mapping out knowledge representation techniques. A. experimented with different step combinations till they found a solution. In this period. GPS). the translations of Russian scientific articles were hasted. Samuel endowed his software with the ability of learning. and achieved successes by simple syntactic manipulations. eventually becoming a worthy opponent on strong human race. The first expert systems were born on the field of medical diagnostics. Lisp is the second oldest programming language still in use today. The application played as a starter level player. This was the era when the first theorem provers came into existence. One of the sources of difficulty was that early softwares had very few or no knowledge about the problems they handled. before defining NP-completeness (Steven Cook. 1965). the logical programming language were born. it was thought that using these softwares for more complex problems is just matter of faster hardware and more memory. Samuel managed to confute the fact that a computer is only capable of doing what it was told to do. John McCarthy created the Lisp programming language. which was built on the computerized realization of a version of the resolution calculus. which proved theorems based on explicitly represented axioms. but the meat is rotten. These systems had (rule-based) knowledge base about the field they handled. AI was unable to beat the „combinatorial boom” – combinatorial explosion and the outcome was the stopping of AI research in many places. which may have been the first software to imitate the protocols of humanlike problem solving. with its 450 rules.” This clearly showed the experienced difficulties. but the flesh is weak” sentence was re-translated. on which an inference engine is executing deductive steps. for example. The MYCIN system.

the yearly income of the AI industry increased to 2 billion dollars. In 1981. industrial process control.) expert systems were used and these were used through a natural language interface. A big class of these techniques includes statistical AI-methods. it made a 40 million dollar yearly saving for the developer company. In 1988. DEC. too. Some of the great achievements of this era is linked to the natural language parsers of which many were used as database-interfaces. Answering the Japanese challenge.prevalent tool in developing expert systems (on medical. by 1988. called R1. DEC's AI-group already put on 40 expert systems and was working on even more. but natural language parsers were implemented in this language. Besides expert systems. which are used in speech. robotics. and by 1986. etc. They are becoming part of our everyday life. but they also gain ground in everyday services. whose research got a boost in the early years of the 1980's from the (re)discovery of neural networks. All in all. when the AI stepped out of the laboratories and the pragmatic usage of AI has begun.and handwriting-recognition. This period brought the brake-through. chemistry. new and long-forgotten technologies have appeared. There had been a mild revolution on the fields of robotics. machine vision. helped to configure computer systems. The hidden Markov-models. On many fields (medical diagnostics. also fall into this category.3 AI BECOMES INDUSTRY (SINCE 1980) The first successful expert system. 2. judiciary. Today. the USA and the leading countries of Europe also started long-term projects with similar goals. and learning. and other scopes). the Japanese announced the „fifth generation computer” project – a 10-year plan to build an intelligent computer system that uses the Prolog language as a machine code. geology. . AI-technologies are very versatile: they mostly appear in the industry.

. Definition 1. position. we need to find a limited number of features and parameters (colour. h n  . We can even define several goal states.C . The functions that describe the state-changes are called operators. To represent a problem.. we will change the different states of the problem's world again and again. c m } ▫ By formalizing a goal condition (in an implicit way): C= {a ∣ goal conditiona } The conditions precondition o a  and goal condition a can be specified as logical formulas. then the states of the problem's world are elements of the set H 1×H 2××H n .1 STATE-SPACE REPRESENTATION The first question is. how to represent a problem that should be solved on computer. weight. price. an operator can't be applied to each and every state.) in connection with the problem that we think to be useful during the solving. we need to define what we mean the solution of a state-space represented problem – as that is the thing we want to create an algorithm for. where Dom o={a ∣ a ∉C ∧ precondition o a } ⊆ A The set C can be defined in two ways: By enumeration (in an explicit way): C={ c1 .O 〉 . (2) k ∈A : is the initial state. (3) C⊆ A : is the set of goal states. where: (1) A : is the set of states. h n (colour: black/white/red. if these parameters are described with the values h 1 . then we say that the problem's world is in thestate identified by the vector  h 1 .. In the followings. As we've determined the possible states of the problem's word this way. temperature: [-20C˚. A state-space representation is a tuple 〈 A .). size. For example. which we will be review deeply in the 3 rd chapter. Naturally.3 PROBLEM REPRESENTATION 3. Every o∈O operator is a function o : Dom o A . Each formulas' parameter is a state a . Now we only need to specify which states can be changed and what states will these changes call forth. O≠∅ . 40C˚]. Furthermore. A≠∅ . (4) O : is the set of the operators. which is a quite universal representation technology. This is called the initial state. so the domain of the operators (as functions) is given with the help of the so-called preconditions.. If we denote the set which consists of values adopted by the i. till we reach an adequate state called the goal state. etc. . . starting from the initial state. we can create algorithms that work on these kind of representations. we have to give a special state that specifies the initial values of the parameters in connection with the problem's world. . During the problem-solving. . ▫ Henceforth. we will learn the state-space representation. After developing the details of a representation technology. k . etc. The concept of a problem's solution can be described through the following definitions: . and the precondition of the operator also has the applicable operator o . parameter with H i . many problem solving algorithms are known in connection with state-space representation..

a ' is directly accessible from a if there is an operator o∈O where precondition o a  holds and o a=a ' . o c for any goal state c ∈C . a ' ∈A are two states. In such cases.o a' Notation: . p) 1 As usual: is the set of the graph's vertices. It can be said. the solution is the sequence of labels (operators) of the edges that formulate this path. a ' is labelled with o if and only if a  a ' . .O 〉 is solvable if k o . and a .… .O 〉 problem for any c ∈C . In this case.an . that all of them explore the state-space graph of the given task in different degree and look for the path that represents the solution in the graph. ▫ ∀ i∈{1. E 〉 .C . it can be interesting to compare the solutions by their costs . In the case of many problems. It can be easily seen that a solution of a problem is nothing else but a path that leads from a vertex k (aka the initial vertex) to some vertex c ∈C (aka the goal vertex).2 STATE-SPACE GRAPH The best tool to demonstrate the state-space representation of a problem is the state-space graph.O 〉 be a state-space representation. Precisely. o Definition 3. The cost of the solution o1 .C . which is a positive integer.Definition 2. ▫ ▫ i 1 2 n−1  Definition 4. a ' ∈A are two states. a 2 .C .and select the less costly (the cheapest) solution. remove mo u n d. 1 n Some problems may have more than one solution. Let 〈 A .o ..2 . p c s ( a1 . the vertices of the state-space graph are the states themselves. Let k o .C . . in general.O 〉 be a state-space representation. The problem 〈 A . k . n−1} : a i  a i1 (any oi ∈O operator) o a o . Definition 6. We label the edges with the operator that allows the direct accessibility. that is.C . the operator sequence o1 . Let 〈 A .  Definition 5. a n ∈A state sequence where a 1=a . 3. k . on is: 1 n ∑ cost oi . In this case. on is referred as a solution to the problem.O 〉 be the state-space representation of a problem. that is cost  o . o Therefore. We have the option to assign a cost to the application of an operator to the state a . we will get to know a handful of problem solving algorithms. . a n =a ' .. the cost of operator applications is uniform. ai  . k . k . . where a . and a . a ' is accessible from a if there is a a 1 . . and we draw an edge between two vertices if and only if one vertex (as a state) is directly accessible from another vertex (as a state). and denote it as cost o a (assuming that o is applicable to a . the cost of a solution is the cost of all the operator applications in the solution. i=1 n Namely. In Chapter 4. Notation: a  a ' . The problem's state-space graph is the graph1 〈 A . Let 〈 A . a ' ∈ E and a . o c in the case of a 〈 A . a =1 for any operator o and state a . is the set of the graph's edges. precondition o a  holds). k . the cost of the solution is implicitly the number of applied operators.

and 8 litres. so it's only their capacities that we certainly know.3 EXAMPLES In this chapter. 3. Let the state be an tuple. The amount of water in the other two jugs at the end is irrelevant. in which we store the capacities of the jugs: ▫ ▫ max=(3. the other two are empty: We can pour water from one jug to another. the set of states is defined as follows: A= { a1. respectively.5. So.1 3 JUGS We have 3 jugs of capacities 3. Initial state: at first. Here are two of the possible goal states: Since there is no scale on the jugs and we don't have any other tools that would help. Initially.8) Set of states: In the states. a3  ∣ 0≤ai ≤max i } where every a i is an integer. we introduce the possible state-space representations of several noteworthy problems. and the goal is to have exactly 4 litres of water in any of the jugs.3. a 2. 5. the initial state is: . So. jug 3 is full all. the middle one 2. we can pour water from jug A to jug B in two different ways: We pour all the water from jug A to jug B . and the largest one 3! Generalize the task to jugs of any capacity: introduce a vector with 3 components (as a constant object out of the state-space). we store the amount of water in the jugs. We fill up jug B (and it's possible that some water will remain in jug A ).3. the 8-litre jug is full of water. There is no scale on the jugs. in which the ith part tells about the jug denoted by i how many litres of water it is containing. Give a number to each jug: let the smallest be 1. the other ones are empty.

3} ∧ i≠ j } Precondition of the operators: Let's define when an operator pour i . the precondition of the operator pour i . j ∣ i . Consequently: pour i . a 3∈ A ∣ ∃i a i=4 } Set of operators: Our operators realize the pouring from one jug (denoted by i ) to another one (denoted by j ). Our operators are defined as follows: O= { pour i .0 . so we define the set of goal states with help of a goal condition: C={a 1 . a 2 . j can be applied to a state a 1 . a 2 . a 2 .2. a3  ! The question is how many litres of water can we pour from jug i to jug j . ▫ Jug j is not filled. where ai −T .k =0. max j−a j Denote this amount with T . otherwise { STATE-SPACE GRAPH The state-space graph of the aforementioned state-space representation can be seen in Figure 2. we can calculate the exact amount to be poured by calculating min a i . a 3=a ' 1 . a ' 3  does the operator pour i . . a2 .3} am . max 3 Set of goal states: We have several goal states. a ' 2 . So. j a 1 . j∈{1. We can also specify that the source jug ( i ) and the goal jug ( j ) can't be the same. a ' 2 . if m=i a ' m= a j T . Since at most max j−a j litres of water can be poured to jug j . j to the state a 1 . a ' 3  . a3  ! It's practical to specify the following conditions: ▫ Jug i is not empty. a3  is: a i≠0 ∧ a j ≠max j Function of applying: Define what state a ' 1 .2. a 2 . j create from the state a 1 . if m= j where m∈{1.

It can be seen that the labels of the bidirectional edges are given in the form of pour i . The state-space graph of the 3 Jugs problem.Figure 2. the red lines depict unidirectional edges while the green ones are bidirectional edges. Naturally. The reason for this is that one pour i .1 Notice that the problem has several solutions. that makes it even more difficult to find a solution. j1− j2 label encodes two operators at the same time: the operators pour i . It can also be noticed that the state-space graph contains cycles . j2 . pour 3. j1− j2 . j1 and pour i . pour 1.3 . j as it was given in the state-space representation. which is the following operator sequence: pour 3. let us use bidirectional edges. which is different from the form of pour i . The bold edges represent one of the solutions. pour 2.1 . bidirectional edges should be represented as two unidirectional edges. The green vertices represent the goal states. pour 2.2 . In the graph.1 . pour 2.2 . . but due to lack of space.

„2”. a3  where a i is the position of disc i (i. (2) The disc which is getting moved to the top of the rod where . P . Our goal is to move all the discs to rod R. So. where ∣ . The discs are denoted by „1”. It's worth to extend the aforementioned condition with another one... a 2 . Q. a3  ! We need to formalize the following two conditions: (1) The disc which is on the top of the rod a which . i. What we need to formalize as a logical formula is that each disc that is smaller than disc which (if such one does exist) is not on either rod a which or rod where . a 3  ∣ ai ∈{P . R}} Initial state: Initially. a state is a vector a 1 . and (6) the discs on the goal rod will be in ascending order by size after the replacing. but can speed up the search (it will eliminate trivial cycles in the state-space graph). R}} Precondition of the operators : Take an operator move which . we store the information about the currently positions (i. Namely: A= { a1 . in this problem. and „R”. namely: C= { R . P  Set of goal states: The goal is to move all the three discs to rod R. R } Set of operators: Each operator includes two pieces of information: ▫ which disc to move ▫ to which rod? Namely: which∈{1. „Q”. or R). O= {move which . we have only one goal state. The initial state of discs can be seen in the figure below: We can slide a disc onto another rod if the disc (5) is on the top of its current rod. namely that we don't want to move a disc to the same rod from which we are removing the disc. This condition is not obligatory. respectively.3. rods) of the discs.where ! Let's examine when we can apply it to a state a 1 .Q .3. where ∈{P . all the discs are on rod P. respectively. a 2 . R . in ascending order of diameter. e. either P.2 TOWERS OF HANOI There are 3 discs with different diameters. We can slide these discs onto 3 perpendicular rods. and „3”.e.3} . We create the state-space representation of the problem. Q . as follows: Set of states: In the states.e. So.: k = P .2. It's important that if there is a disc under another one then it must be bigger in diameter. We denote the rods with „P”. a 2 .

j2 . otherwise { where i∈{1. a 2 . j1− j2 refers to both the operators move i . not just the one that changes! STATE-SPACE GRAPH The state-space graph of the aforementioned state-space representation can be seen in Figure 3. the optimal (shortest) solution of the problem is given by the . and their labels can be interpreted as in the previous chapter: a label move i . then we can apply it to this state. a3  . if i=which ai . a ' 2 . the precondition of the operators is: a which ≠where∧∀ i  iwhich ⇒ a i≠a which ∧ ai ≠where  Function of applying : Take any operator move which . We have to formalize that the disc which will be moved to the rod where . As it can be clearly seen in the figure. Figure 3. all of the edges in the graph are bidirectional.3} Important note: we have to define all of the components of the new state. We have to formalize that how the resulting state a ' 1 . Thus: move which .where a 1 . a ' 3  . a 2 . a ' 3  will look like.2.Thus. Naturally. j1 and move i . while the other discs will stay where they currently are. The state-space graph of the Towers of Hanoi problem. a ' 2 . where a ' i= where . a 3=a ' 1 .where ! If the precondition of the operator holds to a state a 1 .

The operators are expecting only one input data: the column index where we want to place the queen in row s . So: A= { a1 . the optimal solution consists of 7 steps (operators). In this way. then another one to the 2nd row in a way that they can't attack each other. a N . we can solve the task by placing the queens on the board row by row.0 . Set of states: In the states we store the positions of the placed queens within a row! Let's have a N -component vector in a state. a 2 .. N is given as a constant out of the state-space.3 8 QUEENS Place 8 queens on a chessboard in a way that no two of them attack each other. the board is empty. In the state we also store the row in which the next queen will be placed. So.3. One possible solution: Generalize the task to a N × N  N 1 chessboard. s ∣ 0ai N . If the value of s is a non-existent row index. . which is only permitted for testing the terminating conditions. Initial state: Initially. 3. namely. then the vector should contain 0 there.1sN 1 } As one of the possible value of s . in which component i tells us to which column in row i a queen has been previously placed. Thus. N 1∈ A } Set of operators: Our operators will describe the placing of a queen to row s . The set of our operators is: O= { placei ∣ 1i8 } Precondition of the operators : Formalize the precondition of applying an operator . The basic idea of the state-space representation is the following: since we will place exactly one queen to each row of the board. So. a N .  . then we have found a solution. If we haven't placed a queen in the given row. on which we need to place N queens. the set of goal states is: C={a 1 .1 Set of goal states : We have several goal states.rightmost side of the large triangle.0 . in step ith we place a queen to row i while checking that it does not attack any of the previously placed i−1 queens. N 1 is a non-existent row index. the initial state is: k =0. we place one queen to the 1st row.

 . that is. It is also important to note that there is no cycle in the state-space graph. s ! In the new state. we only need to make the following modifications: ▫ write i to the sth component of the state. . a 8 . otherwise s1 . s '  which the operator place i will create from a state a 1 . we need to examine if the value of i was in the state before the sth component. and ▫ increment the value of s by one. a 8 .. where a 'm s' STATE-SPACE GRAPH = = { i . if m=s a m . a 8 . e. So. s =a ' 1 . a ' 8 . . . where 1m8 Function of applying : Let's specify the state a ' 1 . the problem has 2 solutions. So: for all m<s : ∣s−m∣≠∣i−a m∣ Thus. i. by carefully choosing the elements of the state-space representation. we have managed to exclude cycles from the graph. .2 . for all m<s : a m≠i ▫ not attacking any previously placed queens diagonally. If these values are equal then the two queens are attacking each other. Diagonal attacks are the easiest to examine if we take the absolute value of the difference of the row indices of two queens. . which will be quite advantageous when we are searching solutions. s ! It can be applied if the queen we are about to place is ▫ not in the same row as any queens we have placed before. s ' . while the column-index is i . . Thus: place i a 1 . the precondition of the operator place i to the state a 1 . Notice that every solution of the problem is N long for sure. where m∈{1. as compared to the original state. The row index of the queen we are about to place is s . a 8 . and then compare it to the absolute value of the difference of the column indices of the two queens.place i to a state a 1 . In this case. a ' 8 . s is: ∀ m  ms ⇒ a m≠i ∧ ∣s−m∣≠∣i−a m∣ .3} The state-space graph of the aforementioned state-space representation for the case N =4 case can be seen in Figure 4.

. The state-space graph of the 4 Queens problem.Figure 4.

On Figure 5. b . in the database we store the graph in an evened tree form (see below). That's we will be using different cycle detection techniques (see below) in the controller. d . The graph contains cycles. The controller usually executes the steps between (1) and (4) iteratively. Operations: tools to modify the database. We can eliminate the cycles from the graph by duplicating the appropriate vertices. for example. for example. On Figure 6. b . Figure 6.4 PROBLEM-SOLVING METHODS Problem-solving methods are assembled from the following components: Database: stored part of the state-space graph. this method may result in an infinite tree. UNFOLDING THE STATE-SPACE GRAPH INTO A TREE Let's see the graph on Figure 5. d paths are multiple. b . we eliminated the edge from s to s by inserting s to everywhere as the child of s . • negative terminating: we appoint that there is no solution. We usually differentiate two kinds of operations: ▫ operations originated from operators ▫ technical operations Controller: it controls the search in the following steps: (1) initializing the database. d and c . (3) selecting and executing an operation. (2) selecting the part of the database that should be modified. s . c . It can be seen on Figure 6. The unfolded tree version. but the multiple paths in the state-space graph do increase the number of vertices stored in the database. s and the path c . we need to filter the duplications on the tree branches if we want the solution seeking to terminate after a limited number of steps. b . After unfolding. a . Although. (4) examining the terminating conditions: • positive terminating: we have found a solution. they do not endanger the finiteness of the search. The c . as we can use either of them to get from c to d . one such trivial cycle is the edge from s to s or the path s . b . As the state-space graph may contain circles (and loops). the c . we can clearly see what . c . The s . so I only give a finite part of it on the figure. A graph that contains cycles and multiple paths. d and the c . Figure 5. c . s cycle appears on the figure as the rightmost branch. d paths are also multiple in a less trivial way. Of course.

For example. although.g. This means that we can't get into a dead end during the search. . These differences will result in problem-solving methods with different features and we will examine the following of these features in the case of every such method: Completeness: Will the problem-solving method stop after a finite number of steps on every state-space representation. we may get into a dead end from which we can't go back to a previous state. Universal problem-solving methods. which is done on the basis of knowledge about the topic by the controller. The cost of this is the more complex database. the loops do not endanger the finiteness of the search. The point of heuristics is to reduce the size of the database so the problem-solving method will become effective. left to right). … . does the problem-solving method produce the solution with the lowest cost? THE CLASSIFICATION OF PROBLEM-SOLVING METHODS The problem-solving methods are classified by the following aspects: Is the operation retractable? (1) Non-modifiable problem-solving methods: The effects of the operations cannot be undone. not on the same branches (like in the case of cycles). what state-space graph will the problem-solving method need to recognize this? We will mostly differentiate the state-space graphs by their finiteness. up to down. This means that during a search. this last thing entails the reduction of the runtime. which is due to the previously mentioned two multiple paths. But it is worth to use some kind of cycle detection technique in the controller if it holds out a promise of sparing many vertices. (2) Heuristic problem-solving methods: By using some guessing. Note that the existence of multiple paths not only results in the duplication of one or two vertices. there is no such thing as „universal heuristics”. but due to their blind. The advantage of such searchers is the simple and small database. (a ' 1 . p ' ) As I already mentioned. a ' n . but the duplication of some subtrees: the subtree starting at b and ending at the vertices appears twice on the figure. in their operations and the functioning of their controllers. systematic search strategy they are ineffective and result in a huge database.multiple paths become in the tree we got: the vertices get duplicated. Optimality: If a problem has more than one solution. A graph is considered finite if it does not contain a circle. On the other hand. what state-space graph do we need for a solution? ▫ If there is no solution. the quality of heuristics is based on the actual problem. These will differentiate from each other in the composition of their databases. the vertex d appears three times on the figure. as we reduce the size of the database on a large scale and we spare the drive. will it's solution be correct or does a solution even exist for the problem? More clearly: ▫ If there is a solution. How does the controller choose from the database? (1) Systematic problem-solving methods: Randomly or by some general guideline (e. (2) Modifiable problem/solving methods: The effects of the operations can be undone. but on different branches. Moreover. THE FEATURES OF PROBLEM-SOLVING METHODS In the following chapter we will get to know different problem-solving methods.

Controller: The controller is trying to execute an operator on the initial state and will overwrite the initial state in the database with the resulting state. It will try to execute an operator on the new state and it will again overwrite this state. Wemention two solutions: (1) Trial and Error method: o is chosen randomly. than to create a (any kind of) goal state. (2) Iteration: (a) Testing: If the current state (marked with ) is a goal state then the search stops. The general layout of non-modifiable sproblem-solving methods: Database: consists of only one state (the current state). In detail: (1) Initiating: Place the initial state in the database. Figure 7. The flowchart of a non-modifiable problem-solving method.4. They are only used in problems where the task is not to find a solution (as a sequence of operators). The features of non-modifiable problem-solving methods: Completeness: ▫ Even if there is a solution. then the search stops. ▫ If there is no solution. We haven't found a solution. then mark it with . it will recognize it in the case of a finite state-space graph. The certain non-modifiable problem-solving methods differ in the way they choose their operator o for the state a . Executing this cycle continues till the current state happens to be a goal state. (b) Is there an operator that can be executed on ? • • If there isn't. Operations: operators that were given in the state-space representation. . finding it is not guaranteed. Their vital advantage is their simplicity. If there is. but to decide if there is a solution for the problem and if there is one.1 NON-MODIFIABLE PROBLEM-SOLVING METHODS The significance of non-modifiable problem-solving methods is smaller. Optimality: generating the optimal goal state (the goal state that can be reached by the optimal solution) is not guaranteed. only in the case of certain problems. A solution exists. Let o(a) ( be the current state. due to their features they can be rarely used.

restart means that after finding a local minimum.(2) Hill climbing method: We choose the operator that we guess to lead us the closest to any of the goal states.1. the list of noted dead ends is empty. we accept the smallest local minimum we have found as the approached global minimum. The non-modifiable algorithms with restart have great significance in solving the SAT problem. the chance for the algorithm to find a solution also increases – provided that there is a solution. then we simply restart the algorithm (RESTART). than the probability of finding a solution approaches 1. restart. Optimality: generating the optimal goal state is not guaranteed. than it will randomly select an operator (bouncing) till it is discovered that the ball will roll back to the same place. The (only) advantage of the random selection is: the infinite loop is nearly impossible. The so-called „random walk” SAT solving algorithms use these methods. the number of restarts is 0. where the ball is always rolling down. In this example. If yes. the noted dead ends. finding it is not guaranteed. but bouncing a bit before stopping at the local minimum. Controller: (1) Initiating: The initial vertex is the current vertex. ▫ If there is no solution. that is.1 THE TRIAL AND ERROR METHOD As it was mentioned above. The non-modifiable problem-solving methods are often pictured with a ball thrown into a terrain with mountains and valleys. then jump back to . It's foreseeable that by increasing the number of restarts.1. Examine the new state we got whether it is in the list of known dead ends. we replenish the task to exclude this dead end (which can be most easily done by replenishing the precondition of the operator leading to the dead end). in the case of the trial and error method. Completeness: ▫ Even if there is a solution. IDEA: • • If we get into a dead end. In the same time. the number of restarts and the number of maximum restarts. 4. This will be the local minimum.2 THE TRIAL AND ERROR METHOD WITH RESTART Database: the current vertex. 4. (2) Iteration: Execute a randomly selected applicable operator on the current vertex. This approach will be more accurate if the number of restarts is greater. but if there is no such option. The non-modifiable problem-solving algorithms that use restarts are called restart algorithms. In order to exclude getting into that dead end. According to this. The magnitude of non-modifiable problem-solving methods is that they can be restarted. it will recognize it in the case of a finite state-space graph. there is no operator we can use for the current state. If the algorithm reaches a dead end. our heuristics chooses the operator that brings to a smaller state in some aspect (rolling down). note that vertex (augment the database). we apply a random operator on the current vertex. We set the number of restarts in advance. If the number of restarts approaches infinity. In the restart method. we throw the ball back again to a random place.

the we put the found dead end into the database. The features of the algorithm: Completeness: ▫ Even if there is a solution. we choose randomly between the two states. • If the number of maximum restarts have been reached. Because the distance between a state and goal state is guessed through a so-called heuristics. let the heuristics be the sum of the distance of the discs from rod R . (a) Testing: If the current vertex is a terminal vertex. then the solution can be backtracked from the data written on the screen. R . and R− R=0 . So in this case. If no. ▫ If there is no solution. So  R . (b) If there is no applicable operator for the current vertex. P  with heuristics 5. we have to choose between two states: we insert either  R . R h( R . Note that for the goal state  R . P  will be the current state. a 2 . The satisfiability of conjunctive normal forms can be most practically examined with this algorithm. P . P  .the beginning of the iteration. from where we again get to  R . ▫ The greater the number of maximum restarts is. P . if we chose  R . that h c=0 ∀ c∈C . The method with restart is called „random walk”. So: Definition 7. k . we will insert  R . a 3 =∑  R−a i  i=1 3 where R− P=2 . Similarly. So: h a 1 . so the current vertex is a dead end: • If we haven't reached the number of maximum restarts. P  with 4. Q . P . P into the database in the next step. Optimality: generating the optimal goal state is not guaranteed. the initial state  P . P  or Q . If the number of restarts approaches infinity. finding it is not guaranteed. P . The heuristics given for the 〈 A .C . and the hill climbing method doesn't say a thing about how to choose between states having the same heuristics. Initially. Next. then write that we have found no solution. R . Note that. 4. .1. We can apply either the operator move 1. and jump to the beginning of the iteration. Q . R)=0 holds. it will recognize it. and the later one will result in  R . the better the chances are to find the solution. The trial and error algorithm has theoretical significance. increase the number of restarts by one. The heuristics is nothing else but a function on the set of states ( A ) which tells what approximately the path cost is between a state and the goal state. then let the vertex we got be the current vertex. give a possible heuristics for this problem! For example. R−Q=1 .3 THE HILL CLIMBING METHOD The hill climbing method is a heuristic problem-solving method. then the chance of finding a solution approaches 1. state-space representation is a The hill climbing method uses the applicable operator o for the state a where h  o a  is minimal. P . Let's see how the hill climbing method works in the case of the Towers of Hanoi! First. The first one will result in the state Q . Q .O 〉 h : A ℕ function. then we would get back to the previous state. The peculiarity of this situation is that the two states have equal heuristics. where we again . P is in the database. P .Q or move 1. R . let the initial vertex be the current vertex. P into the database. P .

In this problem. We have to say that we need to be quite lucky with this heuristics for hill climbing method even to stop. The basic idea of backtrack search is to not only store the current vertex in the database. a more sophisticated heuristics might ensure this. then the algorithm stops because it haven't found a solution. that is. Another solution is to expand the database with the list of forbidden states. 4. Going on this way. we meet a similar situation in the state  R . R with equal heuristics. then the search can go on a hopefully not infinite branch. it notes the explored dead ends. We would again run into infinite execution with the first one. R and  R . Namely. we delete it from the registration stored in the vertex. .step to  R . Without this.4 HILL CLIMBING METHOD WITH RESTART The hill climbing method with restart is the same as the hill climbing method with the addition that we allow a set number of restarts. but there is no warranty for the existence of such heuristics. except if the heuristics has a random part. we have to see that without storing the past of the search. but all the vertices that we used to get to the current vertex. The easiest method is to change the state-space representation in a way that we delete the current state from the set of states if we run into a dead end. If it reaches the maximum number of restarts and gets into a dead end. then we step back to the parent vertex of the current one and try to another direction from there. and so on till the end of times. We restart the hill climbing method if it gets into a dead end. P  . in every vertex we have to register those operators that we haven't tried to apply for the state stored in the vertex. The (only one) goal state is known. Maybe. P now. The learning can happen in many ways. This means that we will store a larger part of the state-space graph in the database: the path from the initial vertex to the current vertex. Q . The great advantage of backtrack search is that the search can't run into a dead end. This special step – called the back-step – gave the name of this method. it's nearly impossible to complete the task and evade the dead ends. R . which has several variations. Whenever we've applied an operator to the state. All in all.1. If we choose Q . It is important for the algorithm to learn from any dead end. besides the stored vertex's state. 4.2 BACKTRACK SEARCH One kind of the modifiable problem-solving methods is the backtrack search. the heuristics would lead the hill climbing method into the same dead end after a restart. P . It is logical that in the database. It is worth to use this method if 1. so it can't run into the same dead end twice. as we can step to the states Q . 2. or the heuristics is not deterministic. Q . Q . P . we also need to store the directions we tried to step to. either it learns. If there is no further step forward in the graph from the current vertex. Note that the Towers of Hanoi is a typical problem for which applying a nonmodifiable problem-solving method is pointless. the goal is to create one given solution and for this a non-modifiable method is inconvenient by its nature.

2. Controller: Initializes the database. Let us denote the state stored in the current vertex with a ! (c) Testing: If a is a goal state then the search stops. in the state-space graph. If it holds then apply o to a and insert the resulted state o a at the bottom of the path stored in the database. Initial vertex = initial state + all the operators are registered. and decides if it stops the search or re-examines the current vertex. The solution we've found is the database itself (as a path). it will find it in a finite state-space graph. Figure 8. Namely. The flowchart of the basic backtrack method. we will call this the current vertex. Test o 's precondition on a . The controller's action in detail: (1) Initialization: Places the initial vertex as the only vertex in the database. (2) Iteration: (a) If the database is empty. We haven't found a solution. (b) We select the vertex that is at the bottom of the path (the vertex that was inserted at latest into the database) stored in the database. then the controller steps back. Operations: ▫ Operators: they are given in the state-space representation. besides o a . executes an operation on the current vertex. ▫ If there is no solution. In the new vertex.1 BASIC BACKTRACK Database: the path from the initial vertex to the current vertex.4. register all the operators. (d) Examines if there is an operator that we haven't tried to apply to a . ▫ Back-step: a technical operation which means the deleting of the lowest vertex of the path stored in the database. The backtrack search we have got has the following features: Completeness: ▫ If there is a solution. is there any more operators registered in the current vertex? • If there is. denote it with o ! Delete o from the current vertex. • If there isn't. . tests the goal condition. it will recognize it in the case of a finite state-space graph. the search stops.

Optimality: generating the optimal goal state is not guaranteed. The operations of the method can be suited as the following stack operations: ▫ applying an operator ⇐ PUSH ▫ back-step ⇐ POP In what form do we register the operators in the vertices? (1) We store a list of operators in each vertex. don't store all the operators in the new vertex's list of operators. .1 . In the case of the 3-mugs-problem. We can save some storage this way. we know that the operators on the left of the operator index in the array of operators have already been applied to the state. only those that can be applied to the given state. we win the following: it's completely enough to store one operator index in the vertices (instead of the aforementioned list of operators). while the ones on the right haven't been applied yet. we store the indices (or references) of the not-yet-used operators. current) vertex. In the vertices. the (constant) array of the operators consists of 6 elements.1 and pour 3. In this way. IMPLEMENTATION QUESTIONS What data structure do we use for the database? Stack. Idea: when inserting a new state. In the vertices. (3) We can further develop the previous solution by that we apply the operators on every state in the order of their position they occur in the array of operators. The advantage of this method is that we store the operators themselves ain one place (there is no redundancy). With this.1 left. we have got the 3rd (uppermost. In this stack. to which we haven't tried to apply any operator. By applying pour 2. This operator index will refer to the operator that we will apply next to the state stored in the vertex. but it can happen that we needlessly test some operators' precondition on some states (as we may find the solution before their turn). (2) We store the operators outside the vertices in a constant array (or list).2 to the initial vertex (at the bottom).2 and pour 3. namely the position (index) of the operator in the above mentioned array. and we only have the operators pour 1.2 . We have tried to apply the operators pour 3. We have got the 2nd vertex by applying pour 3. 3 vertices of the 3-mugsproblem can be seen. we only store the operator indices.

We represent the stack used by the algorithm step by step. and the algorithm is strictly using the operator with the highest priority. the basic backtrack search will get into an infinite loop. The number of operator executions depends on the order of the operators in the operators' array. This happens because we have assigned a kind of priority to the operators. where pour 2. P . The precondition of the back-step can be very easily defined: if the operator index stored in the current vertex is greater than the size of the operators' array. P  while filling up the stack.1 was the last one (the 3 rd operator). To the 2nd vertex. as sooner or later the search will stuck in one of the cycles of the state-space graph. . then we step back. In the upper left part of the figure. the operators' array can be seen. Next time we will try the 4th one. noting that next time we will try to apply the operator with index 1. On Figure 9. so its operator index refers to a non-existent operator. EXAMPLE: In case of the Towers of Hanoi problem. As it can be seen.We haven't applied any operator to the current vertex. we show a few steps of the search. P . and we also show the travelled path in the state-space graph (see Figure 3). we will step back and forth between the states  R . so let its operator index be 1. P  and the Q . we tried to apply the operators in order. We have tried to apply all the operators to the initial vertex.

we try to expand the number of state-space graphs that the algorithm can handle.2 BACKTRACK WITH DEPTH LIMIT One of the opportunities for improving the basic backtrack method is the expansion of the algorithm's completeness. In a state-space graph it means that we traverse it only within a previously given (finite) depth. If the database gets „full” in this sense . We get to know two solutions for this: we will have the backtrack method combined with cycle detection in the next chapter. 4. The basic backtrack method only guarantees stopping after a limited amount of steps in case of a finite state-space graph. In implementation. We achieve this with a simple solution: maximizing the size of the database. The cycles in the graph endanger the finite execution. it means that we specify the maximum size of the stack ain advance. and a more simple solution in this chapter.2. The basic backtrack in the case of the Towers of Hanoi.Figure 9. so we have to eliminate them in some way. Namely. that does not eliminate the cycles entirely but allows to walk along them only a limited number of times.

The flowchart of the backtrack method with depth limit. which means deleting the vertex on the top of the stack and applying the next applicable operator to the vertex below it. which is 7. then we take a back-step.during the search. Optimality: generating the optimal solution is not guaranteed. EXAMPLE On figure 11. At the 7th step of the search. we have set a depth limit. then it will recognize this fact in the case of any state-space graph. P . Note that if we set the depth limit lower than 7. But if we chose the limit too small. As it is clearly visible. then the algorithm would not find a solution! . backtrack with depth limit does not guarantee a solution. but it can also be seen that it will take tons of back-steps to head forth the goal vertex. In this sense. Let us continue the search from the point where we finished it on Figure 11. P  . Figure 10. namely the constantly duplicating of the states  R . We expand the back-step precondition in the algorithm: if the size of the database has reached the limit . The resulting backtrack method's features are the following: Completeness: ▫ If there is a solution and the value of the limit is not smaller than the length of the optimal solution. then the algorithm steps back. ▫ If there is no solution. The depth limit is indicated by the red line at the top of the stack on the figure. P  and Q . then the algorithm finds a solution in any state-space graph. P . a back-step happens. the stack's size reaches the depth limit. So let us specify an integer limit 0 . then the algorithm doesn't find any solution even if there is one for the problem. the search gets out of the infinite execution.

That is. Backtrack with depth limit in the case of the Towers of Hanoi.2. It can be achieved by introducing an additional test: a vertex can only be inserted as a new one into the database if it hasn't been the part of it yet.Figure 11. another method is the complete elimination of the cycles in the state-space graphe.3 BACKTRACK WITH CYCLE DETECTION For ensuring that the search is finite. . any kind of duplication is to be eliminated in the database. 4.

P  . so we don't put them in again. P created by them are already in the stack. Among the move attempts from the state Q . which creates the state Q . It's worthwhile to note that although the algorithm cleverly eliminates cycles. there are operators through1. P . ▫ If there is no solution. This is how we reach the operator through2. R . It is quite an expensive amusement to scan through the database any time a new vertex is getting inserted! EXAMPLE In Figure 13. It's important to only use this cycle detection test in our algorithm if we are sure that there is a cycle in the state-space graph. P . it reaches the goal vertex in a quite dumb way. but the states  R . the algorithmfinds it in any state-space graph. . the algorithm realizes this fact in the case of any state-space graph. we can follow a few steps of the backtrack algorithm with cycle detection. P  and  P . R and through1. This is the spirit the search is going on. The resulting problem-solving method has the best features regarding completeness: Completeness: ▫ If there is a solution. R . starting from the last but one configuration in Figure 9. so the solution it finds will be far from optimal. that is not part of the stack yet. Optimality: finding the optimal solution is not guaranteed. P . The flowchart of the backtrack algorithm with cycle detection. P .Figure 12. eliminating the duplications in the stack completely. P . The price of all these great features is a highly expensive additional test.

Backtrack with circle-checking in the case of the Towers of Hanoi. we are going to use a version of the backtrack algorithm with . So. then steps back and continues the search. but will make a copy of the database (the vector called solution will be used for this purpose).Figure 13.4 THE BRANCH AND BOUND ALGORITHM We can try to improve the backtrack algorithm shown in the last chapter for the following reason: to guarantee the finding of an optimal solution! An idea for such an improvement arises quite naturally: the backtrack algorithm should perform minimum selection in the universe of possible solutions. As we don't want to traverse the whole state-space graph for this. the algorithm will not terminate when it finds a solution. It follows that the search will only end if the database gets empty. 4.2.

The basic difference compared to the backtrack algorithms is that we not only store one branch of the state-space graph in the database. we give the description of a general tree search problem-solving method That . The Branch and Bound algorithm is usually combined with cycle detection. Whenever finding a solution (and storing it in the vector solution ).depth limit in which the value of the limit dynamically changes. The features of the Branch and Bound algorithm: Completeness: ▫ If there is a solution. The programmers in most cases set the value of limit to the largest representable integer value. the variables solution and limit must be initialized. 4. the limit 's new value will be the length of the currently found solution! So. ▫ If there is no solution. the algorithm realizes this fact in the case of any state-space graph. The flowchart of the Branch and Bound algorithm. In the next chapter.3 TREE SEARCH METHODS Another large class of the modifiable problem-solving algorithms is the class of the tree search methods. It's obvious that the resulting backtrack algorithm – which is called the Branch and Bound algorithm – finds an optimal solution (if the given problem has a solution). Optimality: finding the optimal solution is guaranteed. in the form of a tree. the search is done simultaneously on several branches at the same time. At the beginning of the search. so we probably get a solution sooner. but a more complex part of it. and the limit is such a great number that is by all means greater than the length of the optimal solution. As a matter of fact. Figure 14. too. An obvious disadvantage of this method is its higher space complexity. which is maybe an optimal solution. The solution is an empty vector at the beginning. we won't traverse the state-space graph more deeply than the length of the last solution. the algorithm finds it in any state-space graph.

(d) Expands v . In details: (1) Initialization: Inserts the initial vertex as the only one into the database as an open vertex. decides either top with the search or to expand another vertex. The vertices of the tree stored in the database are divided into two classes: ▫ Closed vertices: previously expanded vertices (c. (b) It selects one of the open vertices. Only open vertices can be expanded. ▫ Open vertices: not yet expanded vertices. Controller: Initializes the database.1 GENERAL TREE SEARCH Database: It is a part of the state-space graph unfolded into a tree. Operation: expansion. which is going to be the current vertex. expands the chosen open vertex. and the A algorithm. which one should be chosen for expansion? . 4. the Best-first method. Expanding a vertex means the following: all the applicable operators are applied to the vertex. according to this. the Depth-first method. (2) Iteration: (a) If there is no open vertex in the database. the Uniform-cost method. Let us denote this vertex v ! (c) If v is a goal vertex. It hasn't found a solution. inserts the created new states as children of v and open vertices into the database.method is not a concrete algorithm. the search stops. Practically speaking. The found solution is: the path from the initial vertex to v in the database. operations). and. Figure 15. The flowchart of tree search methods. the search stops.f.3. There are only three not completely fixed elements in the aforementioned general tree search method: ▫ at (2)(a): If there are several open vertices in the database. but includes the common components of the later described other algorithms – the Breadth-first method. all the children vertices (in the state-space graph) of the given vertex are created. then reclassifies v as a closed vertex. tests the goal condition.

So. How to apply cycle detection? If we only perform the scanning on the current branch. The vertex to be expanded would be taken from the list of open vertices (at the beginning of the list). which does not have any parent). it would be moved into the list of closed vertices. On the one hand. after a vertex has been inserted into the database. this is a quite costly method. This is a less costly procedure than the previous one. a cycle detection must be included by all means. This solution implies that we always have to scan for the open vertex we want to expand in the tree. we introduce the most important such problem-solving methods. and we examine which one is worth to use for what kind of problems. since there won't be two identical states in the database. it would be practical to store the vertices in one more list. If the graph contains only a few multiple paths. we are also looking for multiple paths). naturally. we only check for cycle and don't exclude multiple paths from the database. and another list for the closed ones. We would use a list for the open vertices. In the next chapters. If the problem's state-space graph doesn't contain any cycles. In this case. ▪ It is matter of by arrangement which open vertex is selected for expansion at (2)(a). ▫ at (2)(d): Should we use any kind of cycle detection technique. Otherwise.. on the other hand. The new vertices created during the expansion would be inserted into the list of open vertices. and hence. then traversing the tree top-down is impossible. shall we insert it again? Two sorts of cycle detection techniques can be used: ▫ We scan the whole database looking for the created state. we have to wait with the testing of the goal condition until the given vertex gets selected for expansion. If we chose the scanning of the whole database (i. But if we store the vertices in the above-mentioned two . then our task is more difficult. Thus. since we have to traverse the whole tree. if we stored. ▪ Only the branch that leads to the current vertex is scanned. duplications substantially slow the search. It's more economic to store a pointer pointing to the vertex's parent. but only one parent (except for the root vertex. and after expansion. Detecting cycles or multiple paths in our algorithm depends on the nature of the solvable problem. then it's sufficient to use the less costly procedure. IMPLEMENTATION QUESTIONS What data structure should be used to realize the vertices of the database? Every vertex should store pointers pointing to the children of the vertex.at (2)(c): The goal condition is tested on the current vertex. How should the open and closed vertices be registered ? One option is to store the information about being open or closed in the vertex itself. if any state that comes into being due to expansion occurs in the database.e. then it's expedient to choose the full scan (since this cycle detecting procedure also checks for multiple paths). but scanning the whole database all the time is quite a costly procedure. which one? Namely. This is the point where the concrete (not general) tree search methods differ from each other. then. but the database may contain duplications. In this case. which means that we test the goal condition immediately (before inserting into the database) for the new states that were created in (2)(d). according to the previous point. the testing can be performed sooner in order to haste the search. since a vertex can have several children. which is impossible due to the parent pointers. the parent pointers. then this can be easily realized. we are not only checking for cycles but also for multiple paths. This requires the use of a vertex list in every vertex. but if it contains many. because of the parent pointers. cycle detection is not needed. and if we do. Fully excluding The entire elimination of duplications obviously speeds up the search. In case of some problem-solving methods.

before every expansion. g  m=g n1 .3.3. in this way. we assign a so-called depth number to each of the vertices stored in the database.3. where s is the initial vertex. In practice. the method realizes this fact in the case of a finite state-space graph. the method finds it in any state-space graph. 4. we get to know the systematic versions of tree search methods. Another opportunity for this is to store the open vertices ordered by their depth number in a list.2. Not to speak of the fact that we can also realize in finite steps if there is no solution. Naturally. Definition 8.lists. in case of certain problems one of the extra tests from Chapter 4. it is the optimal solution. the list of open vertices functions as a data structure where the elements get in at the end and leave at the front (when they are being expanded). IMPLEMENTATION QUESTIONS How to select the open vertex with the smallest depth number? One option is to store the vertex's depth number in itself. it causes difficulties when the problem to be solved has long solutions.. Although the breadth-first method finds a solution in finite steps without any cycle detection techniques (assuming that there is a solution). Testing: can be performed sooner. The depth number of a vertex in a search tree is defined the following way: g  s =0 . the different levels (vertices with the same depth) of the tree are created breadth-wise. The breadth-first method selects the vertex with the smallest depth number to expand. ▫ If there is no solution. where m is a child of the vertex n . then this kind of cycle detection can easily be performed: we need to scan both the list of the open vertices and the list of the closed ones.e. Optimality: finding the optimal solution is guaranteed. the most simple data structure to store open vertices in is a queue. The cheapest way of guaranteeing ordering is to insert each new vertex at the correct position (by its depth number) into the already-sorted list. This is where the method's name comes from. .1 should be added. It can be easily seen that. since we can substantially lower the number of vertices that are inserted into the database.2 SYSTEMATIC TREE SEARCH On page 21. ▫ ▫ Completeness: ▫ If there is a solution. It's easy to notice that. which means a lot of vertices in case of certain problems. since finding them can be extremely time-consuming. and. For the exact description. it selects randomly). has a cost: all the levels of the tree must be generated in breadth-wise.1 BREADTH-FIRST SEARCH The breadth-first method always expands an open vertex with the smallest depth (if there are more than one such vertices. This.. only after a level has fully been created we go on with the next level. So. we can see a classification of problem-solving methods that differentiate systematic methods and heuristic methods. this is worth only in connection with problems with frequent cycles (and multiple paths) in their state-space graphs. I. to look for the vertex with the smallest such number in the list of open vertices. By this method. 4. if the breadth-first method finds a solution. of course. i. In this chapter.e. new vertices will always get to end of the list.

It's interesting to follow on Figure 2 how this search tree look like according to the state-space graph. In the search tree. of course. The yellow edges are the ones (besides the red ones) that would be eliminated by a complete detection of cycles and multiple paths (namely. which filters the duplications on branches. a search tree generated by the breadth-first method can be seen. by scanning over the database). in the case of the 3-mugs problem. or at least it does in the case of the 3-mugs problem. 4. we may quickly find goal vertices even if they are deep in the tree.2. has a price: there's no guarantee that the found solution is . The red edges would be eliminated by a cycle detection technique. we illustrate the open vertices with ellipses. Figure 16.EXAMPLE In Figure 16.3. The 3 Jugs problem solved by the breadth-first method. it selects randomly). This quickness. The search tree given here has been built by the breadth-first method without either cycle or multiple path detection. thus. The result of this method is that we don't need to generate the tree's levels breadth-wise. and the closed vertices with rectangles. It can be seen that such a cycle detection technique reduces the size of the database radically.2 DEPTH-FIRST SEARCH The depth-first search method expands an open vertex with the highest depth number(if there are more than one such vertices. for a few steps.

usually find a solution sooner. in the case of depth-first search it's practical to store them in a stack. Both explore the state-space graph „in depth”. ▫ If there is no solution. the method realizes this fact in the case of a finite state-space graph. Optimality: finding the optimal solution is not guaranteed. If our goal is to find an optimal solution. Testing: can be performed sooner.optimal. IMPLEMENTATION QUESTIONS How to select the open vertex with the highest depths number? While in the case of breadth-first search. The depth-first method. we introduce the search tree of the 3 jugs problem generated by the depth-first method. if the goal vertices are rare and their depths is small. So.1 (namely by scanning the database before inserting a new vertex) the method “runs into” a state only once during the search. but it also depends on the state-space graph. while the vertices excluded by multiple path detection are yellow. EXAMPLE In Figure 17. The vertices excluded by cycle detection are red. depth-first search is (generally) out of the question. of course) and none of them is for finding the optimal solution. compared to the breadth-first method. The question is: what extra service does the depth searcher offer compared to the backtrack method? What do we win by not just storing the path from the initial vertex to the current vertex in the database. the method finds it in a finite state-space graph. open vertices are stored in a queue.3. both are complete if the graph is finite (if we don't use a cycle detection technique. but by storing all the previously generated vertices? The answer is simple: we can perform a multiple path detection. The similarity between the depth-first method and the backtrack method is striking. then the breadth-first method is better. Completeness: ▫ If there is a solution. then the depth-first method is the better choice. by combining the depth-first method with the cycle and multiple path detection techniques detailed in Chapter 4. Since the state-space graph of this problem is not finite (there are cycles in it). If the graph contains many goal vertices. Such a search is impossible with the backtrack method. or they are deep in the tree. using some kind of a cycle detection technique is a must. .

we have to introduce the following concept (like the depth number defined on Page 37): Definition 9. and the cost of a solution equals to the sum of the costs of all the operators contained by the solution. that is why we need the uniform-cost method. ▫ g  m= g ncost o n . 4. the . In case of the breadth-first and the depth-first methods.Figure 17. The 3 Jugs problem solved by the depth-first method. Naturally. not all problems are such. ▫ Notice that the depth number used by the breadth-first and depth-first methods is actually a special vertex cost.3. where we have got m by applying the operator o operator to n .2. where s is the initial vertex. Let's recall the concepts and notations from Page 11: the cost of an operator o applied to a state a is denoted by cost o a . To describe the search strategy of the uniform-cost method.3 UNIFORM-COST SEARCH The uniform-cost search is used for problems whose operators has some kind of cost assigned to. So. in the case when cost o n=1 for every operator o and every state n . The cost of a vertex in the search tree is defined as follows: g  s =0 . The uniform-cost method always extends the open vertex with the lowest cost. we did not assign a cost to operators.

fortunately. In this case. 4. The properties of the uniform-cost method: Completeness: ▫ If there is a solution. we have a problem when m is closed. If m is closed. Several solutions exist for the problem of closed vertices. any newly explored path to m is more costly than the one in the database. the g-values of the vertices occurring in the subtree starting from m must be updated (since all of their g-values originate in g  m ). Optimality: finding the optimal solution is guaranteed. ▫ If there is no solution. g  m g ncost o n : We managed to create m on a more optimal path. we haven't put it in again). A problem arises with such a chaining when m already has children in the tree. and update the cost assigned to m (its g-value) to g  ncost o n . the new vertex is not added to the database. Then change the vertex m stored in the database to the new vertex! ▫ This last operation can be easily done: set m tot be chained to n . As we wrote earlier. the method finds it in any state-space graph. g (m)≤g (n)  g (m)< g (n)+cost o (n) Namely. and we have to insert it into the database! Assume that m is already in the database! In this case. In case of the breadth-first and depth-first methods. namely the ability to guarantee . but. in case of the uniform-cost method.following fact can be appointed: the breadth-first method is a special uniform-cost method. Testing: can't be performed sooner. IMPLEMENTATION QUESTIONS How to select the open vertex with the lowest cost ? Unfortunately. and to sort the list of open vertices by cost. neither a stack nor a queue can be used to store open vertices. we don't need any in case of the uniform-cost method.3. since such a modification endangers the finding of an optimal solution. This is guaranteed by the following statement: Statement 1. the sorting of the database should be guaranteed by insertion sort. but with the uniform-cost method. where the cost of every operator is one unit. since the problem of the closed vertices can't occur with the uniform-cost method. in this case. there are two possibilities according to the relation between the cost of m (denoted by g  m ) and the cost of the new vertex (which is actually g  ncost o n ): ▫ g  m≤ g ncost o n : The database contains the new vertex with a more optimal path (the cost is higher). In other words. since. Proof. then g  m is optimal. If a vertex m is closed in the database of the uniform-cost method. then we expand it sooner than the currently expanded vertex n . including the most important one. cycle detection was simple (if a vertex has already got into our database. it gets more complicated. This is called the problem of closed vertices. The only solution is to register the costs within the vertices.3 HEURISTIC TREE SEARCH The uniform-cost method and the breadth-first method (which is a special uniform-cost method) have very advantageous features. the method realizes this fact in the case of a finite state-space graph. Examine the situation when the vertex m is created as the child of the vertex n . So.

In a state-space graph. The uniform-cost method is a so-called systematic method. The search tree stored in the database can consist of a lot of vertices. and this causes the necessity of generating so many vertices. that tells is which child of a vertex should we select to go on with the search. since if it existed then we could know the solution in advance. If we think of a search tree: the heuristic value of a vertex approximately gives the cost of a path to one of the goal vertices. the method finds it in any state-space graph. as we have already introduced it in Chapter 4.optimal solution. we try to enhance our searchers with some kind of foresight. Notice that it is really an estimation. Of course.3. With this number we estimate the total cost of the operators that we need to use to get from a given state to a goal state.1 BEST-FIRST SEARCH The best-first method is a tree search method which selects the open vertex with the lowest heuristic value to be expanded. For example. the heuristic value of a vertex estimates the number of edges that leads into one of the goal vertices. the number of the vertices in the search tree is exponential in the length of the solution. This is caused by the large database. ▫ If there's no solution. The point is that the heuristics as a function assigns a natural number to every state. that is it uses some „blind” search strategy.3. so why bother with searching for it?! The definition of heuristics is quite simple. it loses on important features as compared to the uniform-cost (and the breadth-first) method.1. where the edges have the same cost (like in the case of breadth-first search). by using heuristics. Completeness: ▫ If there is a solution. to wire our own human intuition up in the method. Anyhow. The tool for this purpose is the so-called heuristics. we hope to reduce the size of the search tree. perfect heuristics does not exist. Such a heuristics is called perfect heuristics and can generate 1d d d = 1n⋅d vertices at most. Heuristics is actually an estimation. in the same time. which is the long execution time. To avoid this. the number of vertices is only linear in the length of the solution. in connection with the hill climbing method. so in the case of such a heuristics. and the length of the optimal solution is n . Optimality: finding the optimal solution is not guaranteed. then the search tree consists of 1d d 2d 3 d n = d n1−1 d −1 vertices at most. that could exactly tell us in every vertex which way (which child) leads us to the closest goal vertex. it would be the best for us to have such a heuristics. 4. so we can't be sure if we are going to the right way (towards a goal vertex) in the tree Anyway. IMPLEMENTATION QUESTIONS How to select the open vertex with the lowest heuristic value? . Testing: can be performed sooner.3. But this has a heavy cost. since we can't expect generating an optimal solution. Exception: if the heuristic value of every state is the same (1 unit). since the subtree starting from the child has not been yet generated. but. The advantage of the resulting method is its simplicity. the method recognizes this fact in the case of a finite state-space graph. Namely. if a vertex in the tree has d children at most.

The f-value of a vertex n is actually the estimated cost of a path leading from the initial vertex to a goal vertex via n (namely a solution via n ).2. the list of open vertices have to be sorted by the heuristic values of the vertices.2 THE A ALGORITHM In connection with the best-first method.Obviously. This means that every descendant of m (all the vertices of the subtree starting from m ) will be regenerated once again with a lower g-value than their current one. This. The possible solutions for this problems are: (1) Traverse the subtree starting from m in the database. we use option (2)! So we allow the algorithm to reclassify closed vertices to open vertices in certain cases. Namely. since the costs of the operators are equal and the heuristics is constant zero. the following statement can be proved: Statement 2.f. But there is a very sensitive point in the A algorithm: the cycle detection.3. The uniform-cost method isn't effective since it only considers the „past” during the search.3. too. However. we have to replace the vertex in the database with the new one (update its parent and g-value)! However. Fortunately. it can happen that the vertex we want to replace ( m ) is closed. so their g-values will be updated by the A algorithm. but it can happen in the A algorithm. since we don't know how many times (or even infinitely) a vertex will be reclassified. However.3. on the other hand. Chapter 4. only „gazes into the future”.3): If the vertex we would like to insert is already in the database. too. the problem of closed vertices could not occur in the case of the uniform-cost method.4. The A algorithm always selects the open vertex with the lowest total cost (f-value) to expand. To describe the search strategy of the A algorithm. hence it doesn't always walks on the logical way. this brings forth the danger of a never-stopping algorithm. and now it is created with a lower cost. we need to use heuristics to solve the efficiency problems of the uniform-cost method. The idea: Combine the uniform-cost method with the best-first method! The resulting method is called the A algorithm. The best-first method. so it has descendants in the database. of course. we need to introduce the following concept: Definition 10. For now. so – as the uniform-cost method is a special A algorithm – the question is what kind of heuristics is needed to completely avoid the problem of closed vertices? Option (3) will be examined in Chapter 4. not learning from the past of the search. too.3. The same can be told as with the uniform-cost method (c. we could see that using heuristics ruined all the good features of the uniform-cost (and the breadth-first) method.3. Notice that the uniform-cost method is a special A algorithm where the heuristic value is zero for every vertex. (3) Avoid the occurrence of the problem of closed vertices! This problem doesn't appear at all in the uniform-cost method. which means that we have to update their g-values. means that the breadth-first method is a special A algorithm. Any vertex of the search tree will be finitely reclassified to open. and update the g-values of the vertices in it! Since we are only using parent pointers due to implementation questions. such a traverse is not possible! (2) Delegate the updating of the values to the A algorithm! We can force this by reclassifying m as an open vertex. The total cost of a vertex n is denoted by f n . 4. . and defined as follows: f n= g nhn  .

a state-space graph can be seen. we followed the operation of the A algorithm in this state-space graph step by step. we know that the cost of every operator is positive.f. Counterexample for the optimality of the A algorithm. by this way. Afterwards. Notice that. According to the definition (c.Proof. the method finds it in any state-space graph. the optimal solution is depicted bold. or 0 for the goal vertex c (absolutely correctly. We put the cost of the edges (as operators) on the edges. in which we put the heuristic values of the vertices aside them: for example. All of these facts imply the truth of the statement. which is not an optimal solution! Figure 18. But what about the optimality of the generated solution? We wonder if the A algorithm really guarantees the generation of an optimal solution? A counterexample in Figure 18 shows that it does not. the method has found a solution with a cost of 6 . since its f-value is 23=5 . since all goal vertices should have a heuristic value of 0 ). it can be clearly seen that the method expands the open vertex a . In the figure. Page 11). It's also obvious that every vertex's g-value has a lower limit: the optimal cost of getting to the vertex (from the initial vertex). 5 for the initial vertex s . the following facts can be told about the completeness of the A algorithm: Completeness: ▫ If there is a solution. On the left side of the figure. In the 2nd step. So the following facts can be told about the optimality (and testing) of the A algorithm: . its g-value decreases at least with  . let's examine the features of the A algorithm! By the previous statement. and it is a goal vertex. The search finishes in the 3rd step. which is lower than the other vertex's f-value of 34=7 . On the right side of the figure. as the method selects the vertex c with its f-value of 60=6 . Denote the lowest such cost with  ! It can be easily seen that during reclassifying a vertex to open. the method recognizes this fact in the case of a finite state-space graph. We put their g-values and h-values (cost and heuristics) aside every vertex. ▫ If there is no solution.

It can be seen that the bigger discs are weighted more in the heuristics. it would have been more practical to mark the discs with numeric values (0. { . a 2 . Let us note that. R . The list of open vertices is sorted by f-value. R .Optimality: generating the optimal solution is not guaranteed. the costs are stored within the vertices. the new (open) vertex can be inserted. We have decided to use the following heuristics: h a 1 . until finding a solution. a3 =28−index a1 −4⋅index a 2−9⋅index a 3 where 0 . EXAMPLE In the Figures 19 and 20. It can also be seen that the subtraction from 28 happens to achieve the heuristic value of 0 in the goal state  R . if a i= P index ai  = 1 . and 2) from the beginning. When inserting a vertex into the database with a lowest cost than the existing one. and 2 to R . if a i=Q 2 .1. if ai =R So. step by step. 1 to Q . in case of the Towers of Hanoi state-space representation. a search tree generated by the A algorithm for the Towers of Hanoi problem can be seen. it's worth deleting the old (closed) vertex. and their f-values originate in these values. and when it's done. How to handle the problem of closed vertices ? It's worth to select the 2nd one from the options described on Page 43. Testing: can be performed sooner. IMPLEMENTATION QUESTIONS How to select the vertex with the low est total cost ? Similarly to the uniform-cost method. we assign an ordinal number to the rods: 0 to P . as the goal is to move the bigger discs to the rod R . since the optimal solution is not guaranteed.

0Figure 19. The A algorithm with the Towers of Hanoi (part 1).

In the 3rd step, we indicated two such vertices which the A algorithm does not add to the database by default, since their f-values are not lower than the f-value of the ones with the same content already in the database. During the further steps, we do not show the other such vertices that are excluded for the same reason.

Figure 20. The A algorithm with the Towers of Hanoi (part 2).

The following conclusions can be drawn:
▫ ▫ ▫

The method founds an optimal solution. The question is if it happened by accident, or the chosen heuristics forced the method to do so? C.f. the next chapter! The problem of closed vertices did not occur during the search. Namely, there wasn't any case when a newly created vertex was found in the database as a closed vertex with a higher f-value. The use of heuristics made the search very insinuating. We went for a non-optimal direction only once during the search: in the 6th step, when the vertex  R , Q , R was being expanded. But one step later, the method returned to the optimal path.

4.3.3.3 THE A* ALGORITHM
The A* algorithm is a variant of the A algorithm that guarantees the generation of an optimal solution. We surely know that the uniform-cost method is such a method, and it is obvious that this advantageous feature is due to the constant zero heuristics. The question is: what kind of (but not a constant zero) heuristics could guarantee the generation of an optimal solution? To describe this accurately, we have to introduce the following notations for every vertex n of the search tree:
▫ ▫ ▫

h * n : the optimal cost of getting to a goal vertex from n . g * n : the optimal cost of getting to n from the initial vertex. f * n=g * nh * n : Consequently, the optimal cost of getting to a goal vertex from the initial vertex via n .

Definition 11. A heuristics h is an admissible heuristics if the following condition holds for every state a :

h a ≤h * a  .
The A* algorithm is an A algorithm with an admissible heuristics. In the followings, we prove that the A* algorithm guarantees the generation of an optimal solution. For this, we will also use two lemmas. Lemma 1. If there is a solution, the following condition holds for the A* algorithm in any given time: The optimal solution has an element among the open vertices. Proof. This is a proof by complete induction. Denote the vertices occurs in the optimal solution as follows:  s= n1 , n2 ,  , nr =c  Before the 1st expansion, the initial vertex s is open. ▫ Basis: Assume that before an expansion, the optimal solution has an element among the open vertices, and the one with the smallest index of these is n i . ▫ Inductive step: Let's see the situation before the next expansion! There are two options: ▪ If we haven't expanded n i right now, then n i remains open.

If we have expanded n i right now, then n i1 has been inserted to the database as open. Lemma 2. For every vertex n that was expanded by the A* algorithm:

f n≤ f *  s . Proof: According to the previous lemma, the optimal solution always has an element among the open vertices. Denote the first such vertex with n i . Since n i is stored in the database on an optimal path: g  ni =g * ni  Denote the vertex that is selected for expansion with n ! Since the A* algorithm always selects the vertex with the lowest f-value for expansion, we know that f n is not higher than the f-value of any open vertices, so in case of f n i : f n≤ f  ni  Consequently,
f n≤ f  ni  =  hn i  ≤ g * ni h * ni  = f * ni  = f *  s g ni 
=g * ni  ≤h * ni 

The fact h ni ≤h * ni  originates in the admissible heuristics. The fact f * ni = f * s arises from the knowledge, that n i is an element of the optimal solution, so the optimal cost of the solutions going via n i actually the cost of the optimal solution, namely f * s  . Theorem 1. The A* algorithm guarantees the generation of an optimal solution. Proof: At the moment of the solution being generated, the algorithm selects the goal vertex c to be expanded. According to the previous lemma: f c ≤ f *  s By considering the fact that c is a goal vertex: f c =g c hc  = g c ≤ f * s  
=0

Thus, in the database, the cost of the path from the initial vertex to c (denoted by g  c ) is not greater than the cost of the optimal solution. Obviously, g  c can't be less than the cost of the optimal solution, thus:

3. Now. nr =n Let n i be the first open vertex among these vertices.g  c= f * s  . Optimality: generating the optimal solution is guaranteed. 4.  . if the following condition holds for every state a : if we get the state a ' by applying an operator o to a . Testing: can't be performed sooner. On the other hand. then h a  ≤ h a ' cost o  a . n2 . and it can be proved that the problem of closed vertices does not occur with this algorithm. thus. ▫ If there is no solution. h ni ≤h ni1cost o n i  . Let us continue with the aforementioned formula! f ni =g ni hn i  = g * ni h ni  ≤ ≤ g * n i h ni1 cost o  ni  = g * ni 1 h ni 1 i On the one hand.4 THE MONOTONE A ALGORITHM In the previous chapter we examined. The following fact can be noted: f n i= g n ih ni  = g * ni h n i Here we exploited that n i is already on the optimal path in the database. This is the consequence of the following theorem: Theorem 2. Namely. Demote the vertices on the optimal path from the inital vertex to n as follows:  s= n1 . we assume that the algorithm hasn't found the optimal path from the initial vertex to n . For any vertex n that was selected by the A algorithm for expanding: g  n=g * n . namely. Assumption: g  ng * n . the method recognizes this fact in the case of a finite state-space graph. The A algorithm with monotone heuristics is called the monotone A algorithm. what heuristics the A algorithm needs to guarantee the generation of an optimal solution. the method finds it in any state-space graph. we exploited that the heuristics is monotone. The uniform-cost method (as a special A algorithm) avoids this problem with its constant zero heuristics. Proof: This is an indirect proof. A heuristics h is a monotone heuristics. Let us summarize the features of the A* algorithm! Completeness: ▫ If there is a solution. we exploited the fact g * ni 1 =g * n icost o n i  . Let us continue with the aforementioned inequality in a similar way! i i f  ni =g  ni h ni  = g * n i h ni  ≤ ≤ g *  ni h ni1 cost o  ni  = g * ni 1 h n i1 ≤ ≤ g * n i1h ni2 cost o n i1 = g * n i2 h ni 2 ≤ ⋮ ≤ g * n r h n r  i i1 . but what can we say in general about the heuristics of the A algorithm? Definition 12. g  ni =g * ni  . we examine what heuristics we need to avoid the problem of closed vertices.3.

the following facts can be told about the optimality and testing of the monotone A algorithm: .  . should have expanded n i instead of n ! The consequence of the previous two theorems is the following: in the monotone A algorithm a cycle detection technique that is looking for the new vertex only among the open vertices is absolutely sufficient. the following relation can be drawn: h n 1 ≤ h n 2cost o n 1 h n2  ≤ h n 3cost o n 2  ⋮ h n r−1  ≤ h nr cost o nr −1 1 2 r −1 Sum the left and right sides of these inequalities! We get the following fact: ∑ hn i  i=1 r−1 ≤ ∑ h ni  i =2 r −1 r r −1  ∑ cost o  ni  i=1 i If we subtract the first sum of the right side from both sides: h n1 −h n r  ≤ ∑ cost o  ni  i=1 i = h * n  Here. the following facts can be told about the completeness of the monotone A algorithm: Completeness: ▫ If there is a solution. The first one is that no goal vertices can be reached from n . Moreover. we exploited that h * n is actually the cost of the path (which is optimal!) leading to c (namely to n r ). since n r is a goal vertex. We prove this fact with the help of the next theorem. the method finds it in any state-space graph. where are we now? f n i  ≤ g * nh n  g nh n= f  n Here we used our indirect assumption. thus. it can be seen that the monotone A algorithm is a special A* algorithm. the method recognizes this fact in the case of a finite state-space graph. When summarizing the features of the monotone A algorithm. Proof: It must be proved for any vertex n that the following fact holds in the case of the monotone heuristics h : h n≤h * n There are two options. Accordingly. since we have got that the f-value of n i is lower than the f-value of n . The other option is that a goal vertex can be reached from n . and denote its vertices as follows: n=n 1 . In this case h * n equals to ∞ by definition. another main question is whether the algorithm provides an optimal solution? Fortunately. n2 . nr =c By the monotonicity of h . If a heuristics is monotone then it's admissible. thus. we even know that h n r =0 . too. so the answer to the previous question is „yes”. Take the optimal path among the paths leading from n to any goal vertex. the relation we want to prove obviously holds. namely: g * ng n . Note that this leads to a contradiction.So. ▫ If there is no solution. Theorem 3. Consequently: h n≤h * n Thus.

The uniform-cost method is a monotone A algorithm. (4) The monotone Aalgorithm is an A algorithm. where the heuristics is monotone.3. (2) The uniform-cost method is an A algorithm. since the constant zero heuristics is also admissible. too. we get more and more specialized algorithms and more and more narrow algorithm classes. .3.5 THE CONNECTION AMONG THE DIFFERENT VARIANTS OF THE A ALGORITHM We have got to know the following facts: (1) The breadth-first method is a uniform-cost method. According to these facts. The uniform-cost method is an A* algorithm. 4. too. Testing: can't be performed sooner. where the heuristics is admissible. As going top-down in the figure. since the constant zero heuristics is monotone. (3) The A* algorithm is an A algorithm. where the cost of the operators are one unit.Optimality: generating the optimal solution is guaranteed. equally. where the heuristics is constant zero. the appropriate relations among the different variants of the A algorithm can be seen in Figure 21.

Even in this book. which was specialized on the endgame of chess (the machine had king and rook. that was able to play through a full game of chess was created by Alan Turing in 1951. Actually. These games are called strategy games. came out in 1944 and gives an exhaustive analysis of game strategies. The computer program. which was built in 1940 and was able to win the Nim game any time. we have to give the following information in some way: the possible states of the game ▫ the number of players ▫ the regular moves ▫ the initial state ▫ when will the game end and who will win (and how much) ▫ what information do the players have during the game ▫ if randomness has any part in the game According to the above criterion. When we try to transplant a game to computer. The chess is a game with huge state-space. had similar qualities. Naturally. The first successes of artificial intelligence researches can be connected to games. such as the Spanish Quevedo's chess machine. those games are challenging to research. the alpha-beta purning. (2) By the role of randomness • Deterministic games: randomness plays no role • Stochastic games: randomness plays role (3) By finiteness • Finite games: all players have finite moves and the game will end after finite moves. this is the cause of the difficulties of creating a chess program. (4) By the amount of information: • Full information games: The players have all the information that arise during the . which was worked out by McCarthy in 1956. All these initial attempts remained isolated till the middle of the 1940s. we can categorize the games as follows: ▫ (1) By the number of players: • 2-player games • 3-player games • etc. There were gaming machines before the appearance of computer science.5 2-PLAYER GAMES Games – sometimes to a frightening amount – amuses the human intellect since the beginning of civilization. The Nimitron. like chess. the minimax algorithm was improved (in order to reduce the state-space) along certain aspects – all of these. deserve more attention. in which the players (either human or machine) have verifiable influence on the outcome of the game. draughts or poker. Neumann and Morgenstern's „Theory of Games and Economic Behavior”. competition is present at every field of life. when the first programmable computers were developed. As time has passed. it was tested with hand simulation against a quite weak human opponent who defeated the program. which is a basic work in this topic. Turing's program never run on a computer. the minimax algorithm plays and articular role and it will become one of the basic algorithm in computer game programs. while the human player had king) and was able to give a checkmate in any situation (if the machine had the first step). Probably it comes from the aspect that a game can be understood as an idealized world in which the opposing players compete with each other and face it.

(2) In every goal state.1. which is the player who will move after player p! 5. In the following.2. (5) By profit and loss: • Zero-sum games: the total of the players' profit and loss is zero. 5. Whoever takes the last match-stick loses the game.game. which maximizes the number of . There are only a few differences in the content: (1) The forms of states are a . finite. deterministic.2 EXAMPLES 5. we store the number of matches in a mound. it must be given exactly that who wins or loses in that given state or the game is a drawn. The state-space representation of a game is formally the same as in Chapter 3. So it is important to define p ' . as these have masked cards. denoting the number of the mounds. full information. Set of states: In the states. B } . The function of applying shows if we apply o to a state a . p  . Let the state be an n -tuple. A number max0 must be also given in advance. (3) Every operator o (invariably) has a precondition of the operator and a function of applying.1 NIM There are match-sticks in n pieces of mounds. so p ∈ { A . we will deal with 2-player. The players will be marked with A and B . p '  will we get. Card games are good counterexamples. The players take turns. than what state a ' . where a is a state of the game and p is the player who is next. The player with the next move can take any amount of matches (at least one and no more than the number of matches in the mound) from a chosen mound.1 STATE-SPACE REPRESENTATION We can use the state-space representation to represent games. p  . zero-sum games. where n0 is a constant out of the state-space.

the previous definition of p ' can be as easily described as: p ' =− p On the other hand. if p=B .2 TIC-TAC-TOE We know this game as a 3x3 Gomoku (Five-in-a-row). So the set of our operators is: O= {remove mound .5). Naturally. p wins Set of operators: The operators remove some matches from the mound. ▫ The value of pcs is actually the number of matches the mound th mound has. On one hand.2. So.matches in the mound. a n . p '  the operator remove mound . pcs can be applied to a state a 1 . . a n . The winner is who puts 3 of his or her marks in a row. p ! Namely: remove mound . the precondition of the operator remove mound . so the player with the next move wins. this is quite useful in the function of applying of the operator. So the set of states is the following: A= {a 1. p ' where a ' i= p'= NOTES { { a i− pcs . Namely: C= {0. e. or in a diagonal line. a n .0 . The outcome of the game can be a drawn if the game board is full but no one has 3 marks abreast. the state must store the mark of the player with the next move.g. The player who takes the last match loses the game. pcs generates from the state a 1 . a 2. a ' n . Set of states: In the states we need to store the whole 3x3 game boards.  . One cell of the 3x3 matrix is 0 if none of the players has put a mark there. a ' n .  . . a n . . pcs to the state a 1 . . 5. p is: a mound 0 ∧ pcs≤a mound Function of applying: Let's formalize what state a ' 1 . So the set of states is defined as follows: . Initial state: The initial state can be any state.B}} where every a i is an integer. pcs ∣ 1≤mound ≤n . this notation of the players will come handy in the negamax algorithm (Chapter 5. if i=mound ai . the players put their own marks in turns. if p=A where 1≤i≤n A B It's more worthwhile to denote the players with numeric values. . an . pcs a 1 . so the only clause is the following: k ∈A Set of goal states: The game ends when all of the mounds are empty. p ∣ ∀ i 0≤a i≤max . p =a ' 1 . 1≤ pcs≤max } Precondition of the operators : Formalize when an operator remove mound . p }⊆A . On the 3x3 game board. in a column. p ! We need to formalize the following conditions: ▫ The mound th mound is not empty. p∈{A . where the most common solution is using the values 1 and −1 . otherwise .. and of course the mark of the player with the next move..

Namely: C=C 1∪C 2 . we give the drawn goal states.2=T 3. p∈{A .3 = p ' ∨ ∃i T 1. So.A= {T 3×3  . p)∉C 1 ∣ ¬∃i .A 0 0 0   Set of goal states: The game may end because of two reasons: ▫ One of the players has 3 marks abreast. drawn In the set C 1 . we specify the winning goal states. in the main diagonal .B} . p {∣ ∃i T i . In the set C 2 . y to a state T .3 GAME TREE AND STRATEGY According to the state-space representation of the 2-player games. p ' wins (see the definition of p ' in the operators' function of applying) and C 2= {(T . y  of T is empty. and in the side-diagonal.i = p ' ∨ T 1.3= p' ∨ T 1.1=T i . B}} . p=T ' . if i= x ∧ j= y .1= p' } .2=T 3. y T .i=T 2. using the method given in . In the condition. j=0 } . As it was the opponent of p last time. if p=A 5.i=T 3. j≤3 { A B . j= p'= { p Ti. p ∣ ∀ i . we specify (in order) the triplets in one row. p ! Naturally. A . y to a state T . y ∣ 1≤x . So. p '  where T ' i . y ≤3 } Precondition of the operators : Let's formalize when can we apply an operator put x . j T i . p is: T x .3=T 2. j ∈{0. the precondition of the operator put x . if p=B . in one column.2 =T i . j . that consists of four rows. The game ends in a drawn if the board is full (and none of the players has 3 marks abreast). where C 1= T . p ! So: put x . otherwise where 1≤i . Initial state: In the initial state the game board is completely empty. Set of operators: Our operators put the mark of the current player into one cell of the board. when the cell  x . it's sufficient to look for triplets of the marks of p ' on the board. j T i . y generates from a state T . ▫ The game board is full. the set of our operators is defined as follows: O= { put x . p '  the operator put x .1=T 2. and player A is going to put his or her mark first! 0 0 0 k= 0 0 0 . y =0 Function of applying: We have to formalize what state T ' .

then we transform the game tree into an AND/OR-graph in the following way: with an arc we conjoin all the edges that start from a vertex in which the opponent of p moves. M )∈V ×P (V ) ∣ M ≠∅ } An n . The edges highlighted with red in the figure could be the moves of a strategy of A . Now. we get the so-called game tree. it needs a strategy. Figure 23. An AND/OR-graph is a ordered pair 〈 V . let us formalize. which represent the fact that player p has no influence on the opponent's moves. M ∈ E hyperedge can be of two kinds: ▫ an OR-edge. By unfolding the state-space graph into a tree. in the case of a given game? Namely. the strategy of p must be prepared for any of the opponent's move. 2 One play of the game is represented by one branch of the tree. so this graph must be finite. if ∣M ∣=1 . The strategy is actually a kind of instruction: which operator should the computer apply to a given state? It would be an interesting question whether does any of the players have any winning strategy. we can see a AND/OR-graph from the point of view of player A . too. So. This also indicates that the graph may not contain cycles. since they unambiguously define the moves of A during the game.2. we only deal with finite games. the individual copies of the vertices (and the subtrees starting from them) should be inserted into the tree. We have conjoined the edges starting from a vertex in which player B moves. 2 Unfolding into a tree actually means the elimination of the multiple paths. thus. For a game program to be able to decide the next move of a player. such a strategy in which the operators are applied following the instructions will always lead to the given player's victory (no matter what moves the opponent takes). The game tree as a AND/OR-graph. where E⊆{( n . we can create a state-space graph.Chapter 3. As we told it earlier. On the other hand. what we mean by and AND/OR-graph and a strategy! Definition 13. . We call the resulting conjoined edges an AND-edge group. If we want to represent the possible strategies of the player p ∈{A . For this. we need to investigate how a given strategy (in connection with either player A or B ) appears in the game tree. and E is the set of the hyperedges. in an exact way. In Figure 23. E 〉 where ▫ ▫ V ≠∅ is the set of vertices. cycles are usually eliminated by the game's rules. B} .

framed in the tree. We call the generalization of the corresponding path concept hyperpath: a hyperpath leading from vertex n to the set M of vertices is a sequence of the hyperedges between n and M . each element of the M is a goal vertex) in an AND/OR-graph from the point of view of p . respectively. that's why we define M as the set of vertices 3. if ∣M ∣1 . . it is easy to formalize the strategy of player p : a hyperpath leading from the initial vertex to a set M ⊆C of vertices (namely. Figure 24. 3 P V  denotes the powerset of the set V (namely. After all of the above. The usual edge concept is equivalent to the concept of an OR-edge. A possible strategy for player A. since it is a special hyperedge that leads from one vertex to exactly one vertex. In Figure 24 and Figure 25. The hyperedge is the generalization of the edge concept as we know. A hyperedge can be drawn from one vertex to any number of vertices.▫ an AND-edge group. a possible strategy for player A and player B can be seen. the set of all the subsets of V ).

e. we label the vertices in the tree bottom-up. and its depths is limited by value maxDepths . then ▫ ▫ it gets the label p if it has a child labelled with p . if it doesn't have a child labelled with p (so. the root) shows the player which has a winning strategy. the label of the initial vertex (i. we need to clarify what we call a winning strategy of player p : such a strategy of p which leads (as a hyperpath) to such goal states in which p wins.1 WINNING STRATEGY One of the seducing goals in connection with a game is to look for winning strategies. Generating the tree this way is the same as the common „trick” applied in reality. in the following way: if p moves in the given vertex. Proof: We generate the game tree.3. playing all the possible move combinations in their heads. it gets the label of the opponent. After this. we generate a part of the game tree. After labelling all the vertices. The most we can do is that before every move of the computer player. 5. and then we label its leaves with A and B . a non-losing strategy can be guaranteed for one of the players. It's worth asking for the value of maxDepths as a difficulty level at the start of our game program. that has been given in advance.4 THE MINIMAX ALGORITHM Since a game tree can be extremely huge.5. then does it for which player? First.. In every game (examined by us) that cannot be a drawn. The value maxDepths specifies for how many moves the program should think ahead. it is an irrational wish for our game program to wholly generate it (and look for a winning strategy in it). Theorem 4. depending on who wins in the given vertex. one of the players has a winning strategy. every children of the vertex is labelled with the opponent's mark). ▫ ▫ . This tree has the following properties: its root is the current state of the game. Does one such thing exist? If it does. In games which can end in a drawn. when the players think ahead for a few moves.

and for maxDepths=2 . starting from the state   0 A A 0 B 0 . A B 0 .In Figure 26. we built up an appropriate subtree for the Tic-tac-toe game. B .

and the worse it is for the player. the lower (negative) the value is. so. while if the player loses. B} ) be defined as follows: ▫ 60 . if the given player wins in the goal state. if he loses. we need two heuristics. we define the heuristics. For example. HEURISTICS How to specify a chosen heuristics? The definition of heuristics introduced in Page 24 must be reconsidered in connection with 2-player games: Heuristic values must be arbitrary integer values. In a given game. it should be a very small value. That's why we are trying to define the „goodness” of leaves with some kind of estimation. The other leaves are not goal vertices.The leaves of the tree built in this way can be of two sorts. the heuristic value should be a very great value. Some of them are goal vertices. the greater (positive) the value is. the heuristic value should be 0 only if the goal state results in a drawn. ▫ In case of a goal state. and we have clear concepts about their usefulness: if the given player wins in the leaf. while h B is the heuristics for player B . h A is the heuristics for player A . The better it is for the given player. we need a heuristics. we haven't expanded them because of the depth limit (most of the leaves in Figure 26 are such leaves). we assigned 0 to every goal states. ▫ Since there are two players. in the case of Tic-tac-toe. then it's a „very bad” one. Memo: in Page 24. let the heuristics h p (where p ∈{A . In the case of 2-player games. then it's a „very good” goal state for him.

The explanation of calculating maximum/minimum is obvious: aktP is always about to move towards the most advantageous state. than we use the (unchanged) weight of a ' . So. the worse it is for the other player.5 THE NEGAMAX ALGORITHM The difficulty of using the Minimax algorithm is that two kinds of heuristics must be defined for a game. in all cases. This connection can be described with simple words. p '  of a vertex a . If p= p ' . In Figure 26.1 is being applied (or advised) by the game program. then we use the negated value of the weight of a ' . p  . that is. If p≠aktP . Consequently. −10 . 0 . This is done as follows: • • If p=aktP . 5. p  of the tree. if the goal state is a drawn. p) . if we are in a goal state. The trick is to find a way to transform the weight of the child a ' . p  is a leaf. and diagonals that p „blocks” (his mark occurs there). then let its weight be h aktP a . then the maximum of the children's weights is assigned to (a . we don't need to take into account which player moves in the root of the tree. as follows: the better a state of the game is for one player. p  . We determine which operator is worth applying in the root of the tree (that is. p . the heuristics for one player can be simply the opposite (negated) of the heuristics for the other player. columns. it means that it has some children. p  is not a leaf. we calculate maximum. where the opponent of p wins.(1) (2) (3) (4) 10 . according to the heuristics of player aktP =B . only one heuristic is absolutely sufficient. It's easy to see that there is a strong relation between the two heuristics. The opponent of aktP is about to do quite the opposite. After assigning a weight to each vertex in the tree. if we are in a goal state. (2) If a . then the minimum of the children's weights is assigned to a . as follows: • • If p≠ p ' . The number of all rows. Let the heuristic value of the state a . p  be calculate by the heuristics for p . The values besides the vertices in Figure 26 are the weights of the vertices. Starting on this track. The weights assigned of its children are used to calculate the weight of their parent. in the case of the human player's move) the operator that (as an edge) leads from the root of the tree to the child with the highest weight. (2) When calculating the weight of a non-leaf vertex a . the operator put 1. THE FUNCTIONING OF THE MINIMAX ALGORITHM Denote the player in the root of the tree (who has the current move) with aktP ! Let us assign a weight to all of the vertices a . to the current state of the game). p  . p '  to determine the weight of a . the state with the highest heuristic value. p '  to determine the weight of a . p  . . p  . by the following way: (1) If a . comes the step why we have arranged all these before. where p wins. who takes his move in the root. we can make the minimax algorithm simpler at two points: (1) When assigning heuristic values to the leaf vertices of the generated tree. Namely: the game program should use (or suggest.

it can be seen the tree of the Tic-tac-toe game.In Figure 27. generated by the Negamax algorithm. In these figures. It is worth contrasting with Figure 26. makes a clever move as he or she is controlling the game to the only one direction where he or she has any chance of winning. it can also be seen that player B . we've chosen the heuristics quite well. . So. this time for maxDepth=3 . who is the currently moving.

the time-demand of the minimax algorithm) will be very large.6 THE ALPHA-BETA PRUNING If we choose a high value for maxDepth . So.5. Eliminating these parts is the main goal of the alpha-beta pruning. 63 . it is worth optimizing the algorithm. based on the observation that it is often unnecessary to generate certain parts of the tree. then the generated tree (and also.

its parent's weight will be not less than 9 . hence. for maxDepth=3 . In Figure 30. We managed to cut 57% of the tree. and it will turn out that we need to generate 20 vertices less. since we have already created a child with a weight of 4 . and calculate minimum in it's parent. and only need to generate 15 vertices. (2) Beta cut-off: It is the same as the alpha cut-off. but in this case. since the weight of the current vertex is irrelevant from this point on. We interrupt the expansing of the current vertex. we calculate maximum in the vertex being currently expanded. The Alpha cut-off. Figure 28. The Beta cut-off.With the alpha-beta pruning. a tree generated by the Minimax algorithm for Tic-tac-toe can be seen. We can see such a situation in Figure 28: the vertex being expanded bears the caption ≤4 . We can see such a situation in Figure 29. Figure 29. Let's see two possible situations: (1) Alpha cut-off: We are calculating minimum in the current vertex. For similar reasons. if the minimum value calculated in the vertex becomes less then the maximum value calculated in it's parent. Count them. the weight of the current vertex will be not greater than 4 . and maximum in it's parent. The parts of the tree that the algorithm doesn't need to create due to the Alpha-beta pruning are in a red frame. We do this in cases when it turns out before completing the expansion that the weight of the vertex being currently expanded will have no effect on the weight of it's parent. we can interrupt the expanding of a certain vertex. so we don't need to generate all the other children of the current vertex. .

65 .

one that is psychological and another that is technical. These requirements scare off many students of the subject before they could see it's beauty. So it's very important to make these algorithms tangible and bring them close. First. it exceeds his or her abilities than he or she secludes himself of understanding it.1 THE PROBLEM The problem has two levels. Of course. For example. 6. as the states of the vertices are changing with time. the recursion and ▫ the dynamic data structures. we can only represent what we want by re-drawing the status of the vertex in the same graph. It is difficult to represent these algorithms on a (white)board.6 USING ARTIFICIAL INTELLIGENCE IN EDUCATION The main topic of artificial intelligence consists of the solution searcher algorithms. as the state of a vertex is constantly changing. which is a usually a list (or a stack. We find a solution to both problems if we call for the aid of multimedia. One great method for this is the visualization of the algorithm. the execution of the algorithm can be seen step-by-step. We are looking for a solution for this problem among others. open or closed. The actual line is coloured. in case of the depth-first search. than the graph's appropriate vertex changes on the left of the figure. As the length of the solution is not determinable beforehand. One suggested solution is to use animated figures. So to summarize. every colour's meaning is listed. the students first face the problem that they need to use their programming knowledge in a complex way. If the student feels that a subject is to complicated. According to our experiences. Representing the main solution searcher algorithms on the board is uneasy. a vertex can be unexplored. than they need to select the appropriate algorithm which is recursive in many cases. a vertex can be unexplored. they need to routinely use: the classes. it's storing requires a dynamic data structure. As there is no space on the board to draw the graph many times. that exceed their abilities. By this way. When realizing the searcher algorithms. If any command changes the state of the vertex. they need to create a state-space representation along with the attached data structures (usually with the help of a State and Vertex class). the use of this aid helps and deepens the understanding of the searcher algorithms. that search for the path from the starting vertex to the goal vertex in a graph. Solution suggestion . or maybe a queue). open or closed. Usually it turns to another colour. This is the psychological problem. in case of the depth-first search. ▫ ▫ The other problem is rather technical. We noticed that the students are afraid of the searcher algorithm as they find them to complicated. the audience easily loses the time-line sequence. that's why our digital notes includes several animations. where the algorithm and the graph can be seen. For example. the audience easily loses the time-line sequence. By this way. As there is no space on the board to draw the graph many times. On the right side of the figure. we can only represent what we want by re-drawing the status of the vertex in the same graph.

as part of the graph or completing it and what kind of data structure is used as a database to store the graph. In an opposing case. It's usage needs such tools that are not always given in a lecture room. This is a heuristics searcher. It's a widely accepted view that the materials thought in any subject is more easily learnt if we use multimedia aids during the teaching. using multimedia has drawbacks too. animations. so the conditions of multimedia are usually given. The representable information changes in the cases of the different solution searcher algorithms. such as: colourful figures. so it's required to define for each of them what information should appear in the animation. we will discuss the differences between the representation of the different searcher types. In the notes. videos. they may break down. interactive programs.Using multimedia in education is not a new thing. One of the most wellknow and most customizable type is the Mountaineer method. The practical part of teaching artificial intelligence usually takes place in a computer laboratory. At the same time. their starting takes time and such problems. than we have to produce this goal state. the current vertex is inserted into the list of forbidden vertices and the search restarts. This is very fortunate in the cases when we are able to give an effective heuristics.1. which means that the algorithm selects on operator for a given state by an estimation given by us. 6. The cause of this phenomenon is that multimedia bring closer and makes tangible anything that was said before in theory. . In the following. In this version we record the list of states that we couldn't use to continue the search and we have to give the maximum number of members of this list. When we are unable to continue the search. but we want to know if there is a solution for the problem. we can use the more simple Trials and Errors method which selects operators randomly. and if the answer is yes.1 NON-MODIFIABLE SEARCHERS We use the non-modifiable searchers in such special cases when our goal is not to produce a sequence of operators that lead to the solution. we demonstrate the build-up of the animation on the Mountaineer with restart method.

. actual vertex is C. we portray it when restarting. The next step will be done from this point and will select the operator leading to vertex E. We know this because the heuristics of the state in the E vertex is smaller. Legend: Notions Not yet visited vertex Current vertex Visited vertex Forbidden vertex/state Heuristics Starting vertex Terminal vertex List of forbidden states Markings blue Deep green green red Number by the vertex There's an S by the vertex There's a T by the vertex In the yellow list under the caption We do not note the number of restarts separately. The searcher's current.On the figure above we can see the searcher in a state when we have inserted the D vertex to the list of forbidden vertices after a restart. if it is necessary. The heuristics of the different states are marked with the number by the vertex.

1. We can use this if we can estimate the step number of the optimal solution well.6. that we use to step forth Markings blue Deep green green Number by the arrow . we have the option to continue the search to another direction if the searcher runs into a dead end. This backstep gives the name of the searcher. On the figure. If we find a terminal vertex. On the figure above we can see the searcher in a state when the database that is used to store the vertices gets full. We can easily read from the figure that as the current vertex is the D and it is located under B. By this. This also represents well the operation of the stack data structure. In this version.2 BACKTRACK SEARCHERS One type of the modifiable searchers is the backtrack searcher. we will step back to vertex B in the next step. where we expend our database outside the current vertex with the list vertices leading to the actual vertex. that is usually used for realizing the database in case of the backtrack searchers. we work with a database of which size is given beforehand. Legend: Notions Not yet visited vertex Current vertex Visited vertex Operator. The database on the figure gets filled with elements from the bottom to the top. you can see the backtrack searcher with length-limit. by the expanded database we have the option to give the sequence of operators that we use on the initial state to get the goal state.

we store more paths at the same time in case of the tree search methods. by this we differentiate between open and closed vertices. The closed vertices are the ones we already extended. that can be represented as a tree. By further extending the database we try to find the solution faster and if it's possible to find the solution with the least steps. A vertex's depth number is given by the steps we need to reach it from the starting vertex.3 TREE SEARCH METHODS These make up the other great group of the modifiable solution searchers. While we stored one path in the case of the backtrack searchers. The different algorithms differ in the method they use to select how to continue the search.1.Starting vertex Terminal vertex The stack representing the database There's an S by the vertex There's a T by the vertex The column right to the graph 6. We do the search on different paths in turns. Going forth in the tree is extending (or extension). Another new notion is the depth number. the others are considered open vertices. .

On the figure above we can see the depth-first method which selects the vertex with the lowest depth number for extending. If more vertices have the same depth, it selects the next vertex for extending randomly or by agreement, like from left to right. The searcher on the figure is currently at extending the G vertex. In the next step, the G vertex will be closed and the K vertex will appear in the list of open vertices and it's colour will change to green too. Legend:
Notions Not yet visited vertex Vertex under extension Open vertex Markings blue Deep green green

Closed vertex Depth number Starting vertex Terminal vertex List of open/closed vertices

red Number by the vertex There's an S by the vertex There's a T by the vertex The list under the given caption

6.1.4 DEPTH-FIRST METHOD
We show the depth-first method in short here.

It is visible that the figure has two main parts. On the left part of the animation we can see a graph in which we are searching for the solution. On the right side, we can examine the algorithm of the depth-first method itself. On the right side of the figure we can see as the algorithm is executed step by step. The actual line is red. If any command changes the state of a vertex, then on the left side of the figure, the appropriate vertex of the graph changes. The yellow vertices mark the not yet visited vertices, a the blacks are the closed ones and the red ones are the open vertices. The starting vertex is the uppermost vertex of the graph, the goal (or terminal) vertex is at the bottom. The numbers in the vertices show the depth number of the vertices. The important fact, that the formation of the graph on the picture or in the animation is not entirely random also have to be mentioned, indeed, it was built that way willingly. It's goal is for the figure to show special cases that are usually problematic with some algorithm of artificial intelligence. Such as the diamond and the loop. To understand the animation, we summarize the depth-first method in short. The depth-first method extends the vertex with the highest depth number (the number in the vertex) of the open vertices (the red ones). If there are more such vertices, it selects randomly. We put a ring around the vertex that is selected for extension. As the result of the extension, the not yet visited (yellow) children of the vertex will be open (red) and the extended vertex will be closed (black). The search ends if we are out of open vertices (than we have no solution), or if the last visited vertex is a goal vertex (in this case we have a solution). According to the above explanation, it's visible that such abstract notions, like open vertices can be

specified: red vertices. The table below pairs the notions and the markings of the animation: Notions
Vertices not yet visited open vertices closed vertices Depth number vertex selected for extension extension/extending

Markings
yellow red black the number in the vertex ringed vertex the yellow off-springs turn red, the ringed ones turn black

6.1.5 2-PLAYER GAME PROGRAMS
2-Player games, like the chess, the draughts or the tic-tac-toe can be represented with a game tree. We can create this by using the game's state-space graph. In case of the 2-player games, each branch of the tree represents a different play of the game. To decide in the certain states that where to step next, namely which operator to use, we need to know which player can step in the turn. According to this, we can built up the And/Or graph, which helps with the path selection.

We can see the Minimax algorithm on the figure above. The algorithm works by generating the game tree from the current state for a given depth. When we are done with this, in the leaves of the tree, namely in the finally generated vertices of the branch, we try to give an estimation about how

the teacher can help in a mediate way with the markings. Everyone can see the animation on his/her own computer or it can be displayed with a projector. as we don't need to use a board to draw a graph. as the subject is too difficult. these devices don't work. It is called psychological difference reduction when a problem that is too difficult to solve is traced back to more simple problems that can be solved. It is a disadvantage that we need a computer or a projector at least to see the animation. we have found a solution for the psychological problem. the explanation is much more intuitive.good the given state is for us. and if the student was unable to follow it. The other disadvantageous phenomenon is that the audience may recall the concepts with their markings during the feedbacks. he or she doesn't need to put so much more energy to use the proper concepts. This is a greater problem. To download the animation. That what do we need to select and when is noted on the right side of the animation. heading for the current vertex up in the tree. Legend: Notions Current vertex Selectable operators Depth limit Player Minimum/maximum selection Markings Vertex with the "akt" caption Caption by the arrows On the left ledge On the right. . our opponent or we. internet connection is required. The animation also solves a technical problem. by the frame 6. Hence. The children elements' minimum or maximum is inserted to the vertex. depending on who has the next step. The algorithm becomes more touchable and visible. but if somebody reached the point to understand the algorithm. if the student gets stuck in his/her train of thoughts. Without electricity. by the frame. Moreover. during the feedbacks.2 ADVANTAGES AND DISADVANTAGES According to the experiences. it will restart automatically. so less will feel that it doesn't worth paying attention. with the use of the markings of the animation. This is the explanation why the use of animations reduces the reluctance of the difficult subject – as it is easier to understand the animation. marked with captions "A" and "B" On the right.

this is not important. If the heuristics is good. Till a certain level. This means that we postpone everything we can of the algorithm's steps till we can. . we introduce the most well-known variations of the graph searcher algorithms. the last super-weapon is the mountaineer – method. Unfortunately. as the solution of the problem will still takes many hours. so we have to strive to have as few operators as possible. But we can easily run into a problem. This helps. A good heuristics can hasten up the search by many degrees. This is systematicy. In the second case. This is by no means an accident. It's even more effective if we use a lazy data structure. In our notes. than we still have a chance. these algorithms are closer to systematic trials and errors. like the loop-checking. Of course. This algorithm is unable to realize if there is no solution. that's all! We have to try many possibilities. but our program doesn't find the solution in time. it is really simple and in this manner it is quick. we showed it how can these be used to solve problems that are difficult in a way and a human can't solve them for the first time. If we don't find the solution fast. we give up the search after a given time. for example by implementing the more effective relative of backtrack. the state-space will be much larger. If the second super-weapon doesn't help either. If the kind reader gets to write artificial intelligence for games that tells what the computer's next step should be. What did we say? The graph search is only systematic tries and errors? Heuristics tell which operator worth trying. only in case of the Minimax algorithm. but it's not the answer of artificial intelligence for the problem. but in many cases. We abandon systematicy and trust in tries and errors. The artificial intelligence's answer is the correct choice of the problem-representation.7 SUMMARY As a summary. another question is how fast do we try these possibilities. At the same time. it makes no difference what order do we try the operators. one operator can be that when the boat brings some monks and cannibals to the other side of the river and a different operator can be when we sit in and get out. and have to cover all the possible cases. we haven't dealt much with heuristics. In such cases. on the other hand. he will have an easy time except for writing the heuristics. the only thing that counts is to find the solution fast if it exists. after a difficulty level. Systematic trials and errors. Yes. A problem can be represented with a very large state-space and with a quite small too. This is when the second super-weapon comes into view: the use of heuristics. the less operators we have the smaller the state-space is. as the state-space is quite small and our fast computer can scan through fast. Have you ever thought? On one hand. not even creating a small state-space will help. which has a very large state-space. except for a few trivial optimizations. the mountaineer – method uses heuristics. the backjumping. Let's see a simple problem: In case of the 3 monks and 3 cannibals problem. what can be done if we start to solve a problem with the techniques learnt here. In the notes. the mountaineer – method with restart. Such variants of the algorithms are called lazy algorithms and the data structures that support them are called lazy data structures. This will be the hardest part and it's up to the heuristics how „smart” the game will be. is that all? Yes. How do we know that if the state-space is small or large? Generally. let's see a short heuristics. we have a good chance for this. we can still try to be tricky by optimizing the operators or the algorithms themselves.

The version we use there is called Random Walk. All kind readers can see that this short train of thoughts has just scratched the surface of artificial intelligence.We use the mountaineer – method with restart to solve one of the most significant artificial intelligence problem. We analyse the SAT problem is details in the Calculation Theories notes. . but at the same time all that is written here gives a strong base for further studies. the SAT problem.

It orders that each and every State class have to contain a IsState function. 8. We don't need to specify the operators to. only the main program. // Is true if the i th basic operator is executable for the inner state. This is not obligatory. 8. This is given by the NumberOfOps function. The set of operators is given in a bit oakwardly way for the first sight. then the result is true. // Checks if the inner state is a goal state. The Equals method need rewriting if we want to use backtrack with loop-checking or depth-first method with loop-checking (with depth-first method. i < state. but the compiler will give ugly warnings instead. i++) // { // AbstractState clone=(AbstractState)state. public abstract bool IsState(). else it's false. the arrays need cloning too. This must be extended by the children too. It would only need it if the inner fields of State would be arrays. The other methods don't need overwriting. which have to originate from the AbstractState class.GetNumberOfOps(). /// </summary> abstract class AbstractState : ICloneable { // Checks if the inner state is a state. then the result is true. else it's false. Fortunately. we have tried and tested them many times.Clone(). // If yes. but we know that we can access them by the so-called super-operator. a IsGoalState function. this only needs an own State class. so they probably work. These classes embody the experience of a few years. // It must be called from a "for" cycle from 0 to the number of basic operators. that matches the goalstate predicate. but we know how many member would it have. If we overwrite the Equals method than we should overwrite the GetHashCode method too. the loop-checking is the default). In such case. public abstract int NumberOfOps().1 SOURCE CODE /// <summary> /// The ancestor of every state class. that matches the State predicate. // The super operator is used to access all the operators. This is what the SuperOperator(int i) method is for. . // Gives the number of the basic operators. public abstract bool IsGoalState(). In the next part of the chapter. we review the different classes emphasizing what needs rewriting and what can stay in it's current form. The Clone method probably won't need rewriting in any children class. For example: // for (int i = 0.1 THE ABSTRACTSTATE CLASS The AbstractState class is the ancestor of every State class. the backtrack searcher and the depth-first method. The classes materialize two algorithms. There is no such set. However.1. It is the same as the state-space representation. neither the Vertex nor the BackTrack (and other) classes need rewriting. at some places the source must be altered if someone wants to use it for another graph search problem. // If yes. Other than this. We call this deep cloning.8 EXAMPLE PROGRAMS In this chapter we review some very cohesive classes.

Else it can't be executed and the state transition must be withdrawn too. If it's true. that we can overwrite without violating the OCP. We know of artificial intelligence that these methods are called operators.2. This class has no function.GetHashCode().SuperOperator(i)) // { // Console. than the operator is by no means executable. That is what we call the operator for. public override int GetHashCode() { return base. only show in general how to create basic operators and operators with parameters and how to connect these in the super operator. but that is the same as the IsState predicate. if the Equals method is overwritten. By programming technique. It is true. Every operator has a precondition. /// It shows how to write the operators and connect them to the super operator. Only the super operator needs to be public. // If there is no problem. this should be overwritten too. public override bool Equals(Object a) { return false. From the example. than the state transitions must be executed. 8. // Clones. A new operator have to start with calling it's precondition. the operator can be executed. more basic operators can be given with different values of the parameters. the operators can be done in many ways. If the cloning is swallow enough.2 HOW TO CREATE MY OWN OPERATORS? We talk about the BlindState class in this chapter. // If deep cloning is required. If this is true. We call basic operators the operators with no parameters or with fix parameters. then we are ready. These must be listed in a switch construct in the super operator. We need it because the effects of some operators must be withdrawn. it is visible that every operator has a true/false returning value. their visibility level can be private. As the basic operators need to be accessed through the super operator. Than we have to check if we are out of the state-space with the IsState predicate. // } // } public abstract bool SuperOperator(int i). // If there is a problem.1 SOURCE CODE /// <summary> /// The BlindState is only here for demonstration. } // Only needs overwriting if we use backtrack with memory or depth searcher. The precondition must get the same parameters as the operator itself.WriteLine("The {1}th for the {0} state" + // "operator is executable".// if (clone. so we've remained inside the state-space. state. which says that the inner state of the instance can only be modified by the methods of the instance. } } 8. // The most simple is to clone the state. // By programming technique. // Else. If the precondition is false. // This executes shallow cloning. } // If the two instances are equal. Maybe the one selected here follows most the encapsulation theory of the OOP. // So. than it does not need overwriting in the child class. if it is executable on the current inner state of the State instance. we return to the original state. /// </summary> abstract class BlindState : AbstractState . than their hash codes are equal too. it will the clone be the state of which we continue the search. this is a hook method. i). They also have a postcondition. else it is false. With an operator with parameters. this may remain the basic implementation. public virtual object Clone() { return MemberwiseClone(). than it must be overwritten.

// Else the inner state transition must be withdrawn // TODO: We need to complete the code here! return false. . but we can calmly differ from it. // State transation: // TODO: We need to complete the code here! if (IsState()) return true. if it is true. private bool op3(int i) { // The precondition must be invited with the same parameters as the operator itself. // The operator returns a true. if (!preOp1()) return false. return true. } private bool preOp2() { // If the precondition is true. // An operator is executable if it's true for the inner state. it returns as true.{ // Here we need to give the fields that contain the inner state. // state transition // // TODO: We need to complete the code here! // // Checking the postcondition. // The operators execute an inner state transition. private bool op1() { // If the precondition is false. // First the basic operators must be written. the operator is not executable. // It helps understanding the code. // Every operator's postcondition is the same as it's SateE predicate. if it's executable and a false if it is not executable. // for the preconditions and for the postcondition after state transition. } // Another operator. if (!preOp3(i)) return false. return false. private bool op2() { if (!preOp2()) return false. if (IsState()) return true. // Every operator has a precondition. // Else the inner state transition must be withdrawn. return true. private bool preOp1() { // If the precondition is true. // // TODO: We need to complete the code here! // // and have to return that the operator is not executable. // This is the first basic operator. The name of the precondition is usually: pre+name of the operator. the operator is executable. // This time one operator equals more basic operators. // Else the inner state transition must be withdrawn // TODO: We need to complete the code here! return false. it returns as true. // State transition: // TODO: We need to complete the code here! if (IsState()) return true. } // The precondition of the first basic operator. } // Let's see what is the situation if the operator has parameters.

0. so we have more lines for them here.0. 8. private bool preOp3(int i) { // If the precondition is true than it return as true.0. /// The meaning of 0 is empty.0.0.1.0.0.0.0. this number must be extended too. case 0: return op1().0. We can see that how the different methods should be realized.0.0. case 4: return op3(1). the upper left corner.0. // We have to insert a new operator here. We can see that every State class must originate from the AbstractState class.0 /// 0. /// We represent the table with a (N+4)*(N+4) matrix. The precondition depends on the parameters.0. /// to the cantina.0. The meaning of 1 is: here is the horse. public override int NumberOfOps() { // we have to return the number of the last case.0.0.3 A STATE CLASS EXAMPLE: HUNGRYCAVALRYSTATE In this chapter we can see a well written state-space representation.0. public override bool SuperOperator(int i) { switch (i) { // We need to list all of the basic operators here.0.0 /// 0.} // The precondition's parameter list is the same as the operator's parameter list.0. it much more easy to write the IsState predicate. which runs from 0 to NumberOfOPs.0 /// 0. case 1: return op2(). /// The cavalry must get from the place of the station.0. return 5. } } 8.0.0. case 3: return op3(0).3. // The operators with parameters equal to more basic operators. // The number of lines depends on the task.0 /// 0. default: return false. } // This is the super operator. // With parameter i.0.0 /// 0.0. only a personal State class must be written. case 5: return op3(2).0.0. /// With the use of the margins. which is in the bottom right corner. As to solve one's own task. we say which operator we want to invite. } } // Returns the number of basic operators. it worth starting from this example or the one in the next chapter. // Usually we invite it from a "for" cycle. the inside part is the table. return true.0. The basic operators can be called through this one. // If we extend the number of operators. /// The outer 2 lines are margins.1 SOURCE CODE /// <summary> /// This is the state-space representation of the "hungry cavalry" problem. the initial state is the following: /// 0.0 .0. /// With a 3*3 table.

* Parameters: * x: x direction move * y: y direction move * The precondition checks if the move is a horse step. int x. // We can also set the size of the board here.0 /// It is visible from the above representation that it's enough to store the position of the horse. // The fields describing the inner state. we execute the // state transation. } // Sets the inner state for the initial state. } public override bool IsState() { // the horse is not on the margin return x >= 2 && y >= 2 && x <= N + 1 && y <= N + 1.-2) meaning: 2 up. if not then return false if (!(x * y == 2 || x * y == -2)) return false. y)) return false. // We start from the upper left corner./// 0. /// </summary> class HungryCavalryState : AbstractState { // By default it runs on a 3*3 chess board.0. y = 2. this. } public override bool IsGoalState() { // The bottom right corner is on (N+1. int y) { // if it's a correct horse step. * Example: * HorseStep(1. return true.0. public HungryCavalryState(int n) { x = 2. returns true if it's executable.x += x. * else it returns false. which on the (2. int y) { if (!preHorseStep(x. /// as it is the horse only that is on the table. * The postcondition checks if we remained on the board. } /* This is my operator.0. */ private bool HorseStep(int x. return x == N + 1 && y == N + 1. } private bool preHorseStep(int x. . // due to the margin.N+1) due to the margin. 1 right. y. N = n.0. // If the precondition is true.0. static int N = 3. // Sets the inner state for the initial state. public HungryCavalryState() { x = 2. So the initial state is (beginning from the upper left corner): /// x = 2 /// y = 2 /// The goal state (I'm going to the bottom right corner): /// x = N+1 /// y = N+1 /// Operators: /// The 8 possible horse steps.2) coordinate y = 2.

2). can't stay in the boat. case 6: return HorseStep(-2. case 5: return HorseStep(2. case 3: return HorseStep(-1. } } 8. /// Initial state: /// 3 monks on the left side. this has to originate from the AbstractState class too. -2). return aa. /// Or rather it's generalization to any number of monks and cannibals. -1). he must get out. 2). if there are more cannibals than monks on either side of the river. /// All of them must be transported to the other side of the river. return false. As all State classes. 1). /// then the cannibals eat the monks. this.this. -2).GetHashCode().4 ANOTHER STATE CLASS EXAMPLE In this chapter we can see the state-space representation of the well-known 3 monks 3 cannibals problem. case 2: return HorseStep(-1. case 7: return HorseStep(-2. 3 cannibals" problem. default: return false.y == y. more than two people can't fit in. } } public override int NumberOfOps() { return 8. we substract the width of the margin from x and y. if (IsState()) return true. /// For this. 1). this. . only a personal State class must be written. /// One man is enough to get to the other side. } public override bool Equals(Object a) { HungryCavalryState aa = (HungryCavalryState)a. than it must be withdrawn. public override string ToString() { return (x-2) + " : " + (y-2).x == x && aa. 8. public override int GetHashCode() { return x. /// Problem: there are 3 monks and 3 cannibals on the left bank of the river. a 2-man boat is available. /// If someone goes to the other side.1 THE EXAMPLE SOURCE CODE OF THE 3 MONKS AND 3 CANNIBALS /// <summary> /// The state-space representation of the "3 monks. As to solve one's own task. than their hasítókóds are equal too. // The postcondition is always equals to the IsState.y -= y. case 1: return HorseStep(1.4. -1). // If the postcondition is not true. } public override bool SuperOperator(int i) { switch (i) { // We list the 8 possible horse steps here case 0: return HorseStep(1. case 4: return HorseStep(2. } // when writing it.x -= x.GetHashCode() + y.y += y. } // If two példánys are equal. /// The problem is. it worth starting from this example or the one in the previous chapter.

/// The boat is on the left side. ml = m.1): 1 monk and 1 cannibal goes to the other side. // the number of monks on the right side int cr.'R'.0): 2 monks goes to the other side. // m monks and c cannibals are only transportable // if the boat's side has at least that many. /// Op(0. it's needles to check it as the super operator invites it according to this. // the number of cannibals on the left side char b. } private bool preOp(int m. int c) { // The boat can carry 2 people. int c) { .3.1): 1 cannibal goes to the other side. else return mr >= m && cr >= c. // Where the boat is? It's value is either 'L' or 'R'. int c) // sets the initial state { this. mr = 0.3) /// Operator: /// Op(int m. if everyone got to the other side return mr == m && cr == c.'L'.3.0) /// The goal state: /// (0. /// Op(1. int mr. /// 0 monks on the right side. cl = c. /// This state is described with the following organized quintet: /// (3. } public override bool IsState() { // true. // the number of monks on the left side int cl. /// 0 cannibals on the right side. // At the end.0. } public override bool IsGoalState() { // true. at least 1 is needed for paddling. // the number of monks all together int c. private bool op(int m. /// </summary> class MonksAndCannibalsState : AbstractState { int m. /// Possible parameters: /// Op(1.c = c. if the monks are safe return ((ml >= cl) || (ml == 0)) && ((mr >= cr) || (mr == 0)). if (m + c > 2 || m + c < 0 || m < 0 || c < 0) return false./// 3 cannibals on the left side.0. cr = 0. // the number of cannibals on the right side public MonksAndCannibalsState(int m. b = 'L'. // the number of cannibals all together int ml.2): 2 cannibal goes to the other side. } // Transports m monks and c cannibals to the other side. /// Op(0. if (b == 'L') return ml >= m && cl >= c.m = m. this.0): 1 monks goes to the other. /// Op(2. int c): /// m monks go to the other side and /// c cannibals go to the other side.

} public override bool SuperOperator(int i) { switch (i) { case 0: return op(0. } } 8. 1). case 4: return op(2. return false.if (!preOp(m. if (b == 'L') { ml -= m.cl == cl && aa.5 THE VERTEX CLASS In this chapter." + cl + ". } public override bool Equals(Object a) { MonksAndCannibalsState aa = (MonksAndCannibalsState)a." + b + ". b = 'R'. } else { ml += m. case 2: return op(1. cl += c. mr = without. mr -= m. cr = without.mr. 0).ml == ml && aa. c)) return false. ml = without. } } public override string ToString() { return ml + ". public override int GetHashCode() { return ml. } if (IsState()) return true. cl = without. b = 'L'.cr.b. // mr and cr are countable. 2).cl. } // If two instances are equal.b == b. case 1: return op(0. default: return false. b = without. we don't need to change this if we want to solve one's one task with backtrack or with depth-first method. MonksAndCannibalsState mentes = (MonksAndCannibalsState)Clone()." + cr. 1). mr += m. so no need for checking return aa.GetHashCode(). cr += c.ml." + mr + ". } public override int NumberOfOps() { return 5. If we want to implement some . Fortunately. cr -= c. than their hash codes are equal too. we get to know the vertex class.GetHashCode() + b.GetHashCode() + cl. case 3: return op(1. 0). cl -= c.

other algorithm. only after that it is ready. } public int GetDepth() { return depth. depth = parent. the vertex's depth and the vertex's parent.Clone(). for recording the path it's enough to store the last vertex of the path. public Vertex(AbstractState initialState) { state = initialState. the vertex contains a state.SuperOperator(i). /// So the vertex represents a whole path till the starting vertex. i < NumberOfOps(). public Vertex(Vertex parent) { state = (AbstractState)parent. As the vertex knows it's parent. It supports extending and with the help of the super operators.state). then we need to modify it so the state will contain the the value of the cost function. it's depth and it's parent AbstractState state. Vertex newVertex = new Vertex(this). } public override bool Equals(Object obj) { Vertex cs = (Vertex)obj. i++) { // We create a new child vertex. this. This means that for the solution. // Going upwards on the parents we get to the starting vertex. } public int NumberOfOps() { return state.NumberOfOps(). Vertex parent. } // Executes all the executable operators. parent = null. } public bool TerminalVertexE() { return state. } public override String ToString() { return state.ToString().depth + 1.Equals(cs. it's enough to return the terminal vertex where the solution leads too. 8. /// </summary> class Vertex { // The vertex contains a state. In this version. // Returns the so created new vertices.IsGoalState(). int depths.state.parent = parent.5. } public override int GetHashCode() { return state. // Constructor: // Sets the inner state on the starting vertex. } // A new child creates a vertex. // It's the inviter's responsibility to invite it with the initial state.GetHashCode(). } public bool SuperOperator(int i) { return state. } public Vertex GetParent() { return parent. instead of an algorithm based on depth. return state. like the uniform-cost method. it's depth and it's parent. // We need to invite an applicable operator. // The depth of the starting vertex is 0 and it has no parent. for (int i = 0. depth = 0. . public List<Vertex> Extending() { List<Vertex> newVertices = new List<Vertex>(). we can use the operators one by one.1 SOURCE CODE /// <summary> /// The vertex contains a state.

// We try the i th basic operator? Is it executable? if (newVertex. It has one abstract method. /// <summary> /// Writes the solution based on a terminal vertex. The solution can be written with the WriteSolution method. /// From this terminal vertex. in other case. protected Vertex GetStartingVertex () { return startingVertex. so we prescribe a constructor too.1 SOURCE CODE /// <summary> /// The ancestor of all graph searchers. startingVertex = startingVertex. // The starting vertex vertex. This must be invited by every children class. but the children classes can request it.SuperOperator(i)) { // If yes. the solution can be made by going up on the parent references. in case of the depth-first method. /// If a vertex is null. else it's null. /// The graph searchers only need to realize the Search method. If we decide on using it. there is an example in the main program for its call. This is what needs to be realized in children classes. Every graph searcher starts searching in the starting vertex. This must be the terminal vertex.Add(newVertex). The Search returns with a vertex. /// It reverses the order of the vertices to write the solution correctly. In case of the backtrack. The returning null value indicates that there is no solution. we add it to the new ones. 8. if a solution exists. its use is not required. the Search. This is only a small aid. public abstract Vertex Search(). /// that leads from the starting vertex to the terminal vertex. } } 8. /// </summary> /// <param name="oneTerminalVertex"> . Al graph searches must originate from this class. } // It's better if the starting vertex is private. it will be the backtrack itself.6. the solution can be made by going up on the parent references in reverse order. /// The solution is given as a terminal vertex. it must be null. // Every graphsearxher starts searching in the starting vertex. if there is a solution or a null if there isn't. namely there is a path in the state-space graph. This class probably won't need modification. } /// If there is a solution. newVertices. /// This will return a terminal vertex. /// It assumes that by going up on the references of the parent vertex we will get to the starting vertex. it will be the depths search. public GraphSearch(Vertex startingVertex) { this. /// </summary> abstract class GraphSearch { private Vertex startingVertex. /// it returns a solution. /// From the terminal vertex. it writes that there is no solution. not even in the case when a new graph searcher algorithm has been written.6 THE GRAPHSEARCH CLASS In this chapter we introduce the common concepts of the graph searcher algorithms. } } return newVertices.

foreach(Vertex cur in solution) Console. bool memory) : this(startingVertex. } } 8. return. than the searcher has depth limit. int limit.Push(curVertex).GetParent(). false) { } // searcher with memory public BackTrack(Vertex startingVertex.the backtrack with depth limit /// .limit = limit. false) { } // searcher with depth limit public BackTrack(Vertex startingVertex. // Returns a terminal vertex. } // The order of the vertices must be reversed. bool memory.memory = memory. It is a well-tested code. These are: /// . We can get to this terminal vertex from the starting vertex. //if it's true. It is a great practice. limit. Stack<Vertex> solution = new Stack<Vertex>(). and if it doesn't work. this. can be written. while (curVertex != null) { solution. 8. } // there is neither a depth limit. 0. memory) { } // The search starts from the starting vertex. curVertex = curVertex. public BackTrack(Vertex startingVertex. /// It contains the three basic backtrack algorithms in one.WriteLine("No solution"). It worth trying to rewrite the below realization to a loop based solution. 0. // If it's not null. nor a memory public BackTrack(Vertex startingVertex) : this(startingVertex. we don't need to even touch this code.7. with memory or the combination of these. /// </summary> class BackTrack : GraphSearch { int limit./// The terminal vertex representing the solution or null. } // It has been reversed. /// </param> public void writingSolution(Vertex oneTerminalVertex) { if ( oneTerminalVertex == null) { Console. . In this. int limit) : this(startingVertex. the searcher has memory. Different constructors can be used to create a backtrack with depth limit.7 THE BACKTRACK CLASS In this chapter we introduce a recursive solution of the backtrack graph search.WriteLine(cur).the backtrack with memory /// The backtrack with branch-limit hasn't been worked out here.1 SOURCE CODE /// <summary> /// Class that realizes the the backtrack graph searcher algorithm. the original is still there to create the hand-in program. Vertex curVertex = oneTerminalVertex.the basic backtrack /// . bool memory) : base(startingVertex) { this.

the terminal vertex must be returned. we introduce the depth-first search in two versions.GetParent(). curParent = curParent.// If there is no such vertex.GetDepth(). // checking the depth limit if (limit > 0 && depth >= limit) return null. } if (curVertex.8 THE DEPTHFIRSTMETHOD CLASS In this chapter. if (memory) curParent = curVertex. } // The operator should be withdrawn on the else-branch. // If any of them can be applied. Is it applicable? if (newVertex. // This will be done only if we apply an applicable operator on it. if (curVertex. Vertex terminal = Search(newVertex). we create a new vertex. then a solution has been found. return curVertex. // the use of memory to remove circles Vertex curParent = null. for (int i = 0. this part is empty. } // The recursive realization of the searcher algorithm. then backstep. the backstep is the "return null". Vertex newVertex = new Vertex(curVertex). if (terminal != null) { // We return the terminal vertex that represents the solution.GetParent(). } // We invite the basic operators here through the super operator. The other version doesn't use it.TerminalVertexE()) { // We have the solution. The first version uses loopchecking. // If it returns with a null value. private Vertex Search(Vertex curVertex) { int depth = curVertex. // We try the ith basic operator. // As we cloned. so it is much faster. else the . i++) { // We create the new child vertex. // and invite myself recursively. public override Vertex Search() { return Search(GetStartingVertex()). // This is required if there was no cloning in the creation of the new child.SuperOperator(i)) { // If yes.Equals(curParent)) return null. // Going back on the parent trail. } } // If we tried all the operators and none of them has given a solution then backstep. // As it is recursive. then we invite myself recursively to the new vertex. while (curParent != null) { // We check if we have been in this state previously. If yes. return null. we try to use the next basic operator one level higher. } } 8.NumberOfOperators(). // If it returns with a value other than null. we have to try the next basic operator. as the original depth searcher uses it too. it returns a null value. The second version should only be used if there are no loops in the state-space graph. i < curVertex. return terminal. // After the backstep.

Count != 0) { // This is the open vertex with the greatest depth. } // The default value of circle checking is true. /// The open vertices are stored in a stack. Stack<Vertex> Open. If this cannot be assumed. return SearchingTerminalVertexFast().8. the Vertex class must be completed with the cost. To achieve this. than we need to test the vertex that we selected for expanding if it is a terminal vertex or not. // We expand this one. where there is a heuristic too. public override Vertex Search() { // If we don't need the circle checking. it may enter into an infinite cycle. It is a well-known fact that the depth-first search should be done with a stack. as we don't need to look for the vertex with the greatest depth. it worth storing the open vertices in a stack. bool circleChecking. a sorted list should be used to find the open vertex with the lowest cost. With the uniform-cost method. This greatly hastens the search. we also need to pay attention to the fact that more paths may lead to one vertex. Due to this. public DepthFirstMethod(Vertex startVertex. true) { } // The is the point where the search for the solution starts. // So we don't need to search for the open vertex with the greatest depth. on the top of the stack.circleChecking = circleChecking. bool circleChecking) : base (startingVertex) { Open = new Stack<Vertex>().Pop(). The A-algorithm. // at the beginning. while the breadth-first search should be done with a queue. // If it is false. List<Vertex> Closed. // at the beginning. if (circleChecking) return SearchingTerminalVertex(). /// This realization of the depth search assumes that the starting vertex is not terminal. According to this. Vertex C = Open. . is even more exciting. the set of closed vertices is empty this. // as in this way. it's enough to test the new vertices. we store the open vertices in a stack.searcher may enter an infinite cycle. Open. the algorithm will be much faster. public DepthFirstMethod(Vertex startingVertex) : this(startingVertex. while (Open. } private Vertex SearchingTerminalVertex() { // Till the set of open vertices is not empty. as it is done in the code below. We advice that the reader would write the breadth-first search based on the code below. it will be the open vertex with the greatest depth. but we only need to store the one with the lowest cost. 8. Both versions assume that the starting vertex is not terminal. the vertex with the greatest depth will be on the top of the stack. It is a bit more complicated task to write the uniform-cost method based on these information.Expending(). only the starting vertex is open Closed = new List<Vertex>(). List<Vertex> newVertices = C.Push(startingVertex).1 SOURCE CODE /// <summary> /// A graph search class that realizes the depth search algorithm. and instead of the stack. Otherwise. // Set of the open vertices. /// </summary> class DepthFirstMethod : GraphSearch { // In the depth search. // Set of the closed vertices. This shouldn't cause any trouble.

but it is not enough for the 3 monks. Another worthy thing is playing with the constants. nor in the set of closed vertices. if (D. It's very important for the starting vertex to get the initial state.Pop().Contains(D)) Open. It may be salient to.9. it is needless to check if D had been among the open or closed vertices.Expanding(). } return null.Add(C). It's also interesting to check that what size of a task will make the problem difficult to solve. Open.Contains(D) && !Open.TerminalVertexE()) return D. } // if there is no circle. // We only pick up the vertices to the list of open vertices // that haven't been included neither in the set of open. the Searching method must be invited. It is visible that it doesn't matter which graph searcher algorithm we use. Closed. 3 cannibals. // The Contains invites the Equals method that was written in the Vertex class.Count != 0) { Vertex C = Open. where the backtrack has plenty of memory. } // This should only be used if we are sure that the state-space graph does not contain a circle! // Else it will probably cause an infinite cycle. These are all the responsibilities of the main program. It worth checking that we can invite different searchers on the same starting vertex and they will end in slightly different results.Push(D).9 THE MAIN PROGRAM In this chapter we can see how the different parts should be merged.Push(D). that the depth-first method may run out of memory with greater problems. if (!Closed. The writeSolution method should be invited on the vertex that was returned by that method. List<Vertex> newVertices = C. 8.TerminalVertexE()) return D. } // We made the expanded vertex to be a closed vertex. private Vertex SearchingTerminalVertexFast() { while (Open. } } 8. it is needless to made C closed. } return null. foreach (Vertex D in newVertices) { if (D. A brave man may rewrite the super operator to use heuristics. It also worth noting that reordering the sequence of operators in the super operator influences the run time greatly. // If there is no circle. We have to give the starting vertex to the constructor of the graph searcher.foreach (Vertex D in newVertices) { // I'm ready if we have found the terminal vertex.1 SOURCE CODE class Program { static void Main(string[] args) . Note that a depth limit of 10 is enough for the example of Hungry Cavalryman.

")."). 3 cannibals problem. true).writeSolution(search. search. search = new DepthFirstMethod(startingVertex.WriteLine("The searcher is a depth searcher with circle checking. true).").WriteLine("The searcher is a depth searcher with circle checking.Searching()).Searching()). search.WriteLine("The searcher is a backtrack searcher with memory and depth limit of 10.Searching()). search = new BackTrack(startingVertex.writeSolution(search. Console.WriteLine("The searcher is a backtrack searcher with memory and depth limit of 15. GraphSearch search.3)). true). true). Console. 10. startingVertex = new Vertex(new HungryCavalryState(4)).writeSolution(search.Searching()). search = new DepthFirstMethod(startingVertex. search.ReadLine(). Console. } } . Console.WriteLine("We solve the 3 monks. search. search = new BackTrack(startingVertex."). Console.writeSolution(search. startingVertex = new Vertex(new MonksAndCannibalsState(3. 15.").").WriteLine("We solve the Hungry Cavalryman problem on a 4x4 board.{ Vertex startingVertex. Console. Console.

pdf (5) CLIPS http://www.ektf.html (6) SWI-Prolog http://www. 2000 (Hungarian title) (2) Pásztorné Varga Katalin. Kovásznai Gergely: Logikai programozás http://aries.hu/~kovasz/logprog/logikai_programozas.BIBLIOGRAPHY (1) Stuart J.hu/~varteres/mi/tartalom.inf. Peter Norvig: Mesterséges intelligencia modern megközelítésben Panem – Prentice Hall.ghg.swi-prolog. Várterész alkalmazásszemléletű tárgyalása Panem.org .unideb.htm (4) Dr. 2003 Magda: A matematikai logika (3) Dr.net/clips/CLIPS. Russell. Várterész Magda: Mesterséges intelligencia http://www.

Sign up to vote on this title
UsefulNot useful