ADVERSARIAL SEARCH : GAME PLAYING
But, first let’s talk about Heuristic Function
Heuristics :
“the study of the methods and rules of discovery and
invention (Polya, 1945)”
Employ heuristics in two basic situations:
1. A problem may have an exact solution, but the
computational cost of finding it may be prohibitive
2. A problem may not have an exact solution because of
inherent ambiguities in the problem statement or
available data
Heuristic Function in Game Playing
First set of
moves
5
Game Playing : A Classical Search Problem
• A set of states (each state includes turn information)
• Initial states (initial state, turn)
• A set of actions (legal moves)
• A successor function which takes a state and an action
and returns all legal states that result
• A set of terminal states or a termination condition
(when is game over)
• Cost or payoff function mapping terminal states to real
number (evaluates game outcome)
Class of Games
• Two person : Max and Min
• Zero-sum : A win for Max is loss for Min and vice versa
• Discrete : Set of states and actions are discrete
• Finite : Set of states and actions are finite
• Deterministic : Next move determined by current position
• Perfect information : Both players have access to game
state, and players move sequentially
Why We Need Game Playing Algorithms
Games cannot be solved using single agent search
algorithms because the presence of an opponent
introduces uncertainty even for games of perfect
information. This is why we only calculate the next move,
not the sequence of moves to victory
Standard assumption : Assume perfect opponent in order
to generate search tree
Games are difficult !. The search tree for chess has about
35100 nodes
Real need for resource bounded optimal algorithms
Generating Game Tree
The Minimax Search Method
Game Tree
A tree consisting of nodes that represent the possible moves
that each player can make in a two-player game.
Ply
A single move by either the player or their opponent.
Move
A set of plies, whereby each player makes a move.
The games that we will be considering are two
player games, so each move will consist of two
plies.
The Minimax Strategy
• Optimal strategy – leads to outcomes at least as good as
any other strategy when playing an infallible opponent.
• Pick the option that minimizes the maximum damage that
your opponent can do
• Minimize the worst case outcome
• Because your skillful opponent will certainly find the
most damaging move
1 3
2 4 5
O
New 1 2 3 4 5
state
O sum 200+5 200+4 200+4 200+3 200+5
O Total 205 204 204 203 205
X total 1 2 1 1 2
204 202 203 202 203
Taking the highest value we see that with this information
the best move to take is the center position in the board.
How can we be sure that this is going to produce best
possible path though ?
O I’am O and it’s my
O X
turn to play
X
Possible moves that I can make from this
state are:
O O O O O O O
O X O X O O X O X O X
X X X O X O X
If I were to choose the first of these possible moves,
then X would have the option of any of these moves:
O O X O O O O O O
O X X O X O X O X
X X X X X X
Score: Score: Score: 0 Score: -99
(200+2)-(600+1) (400+3)-(300+2)
O I am O and it’s my turn to play!
O X
X
Score: 1
Maximizing Ply: The largest of the children’s values is
propagated upwards
OO X O O O O O
O X O X O O X O X O X
X X X X O X O X
Score: Score: Score: Score: Score:
(400+2)-(400+1)
-399 -399 -399 -399
O I am O and it’s my
O X
turn to play!
X
Score: 1
Maximize:
O O O O O O O
O X O X O O X O X O X
X X X O X O X
Score: -399 Score: 1 Score: -399 Score: -399 Score: -399
Minimize:
O X O X O X
O O X O O OX O O O OX O X O X O X O X O X
O X X O X O X X O X O O X O O X O X O
X O X O X
X X X X X X O X
201 302 Score: 0 Score: ? Score: 101Score: -399Score: -199 Score: ?
-399 101
O O O
O O O O O O O O O O O O X X O X O X
O X O X O X O X O O X O O X X O X O X X O X X O X
X X X X X X X X X X X X O X
0 -99 1 102 Score: ? Score: ? Score: 2 Score: ? Score: ?
Minimax Algorithm (limited look-ahead +
scoring)
Maximizes the heuristic of Max under the assumption
that Min will play perfectly to minimize it
Max
Min
4 10 5 3 9 17 7 2 1
Mimimax Algorithm (contd.)
Max 4 Value of game tree = 4
Min 4
4 10 5 3 9 17 7 2 1
Marked path is called the principal variation. All node on it
have the value at the root
Two Player Games Algorithms: MiniMax
function MINIMAX(N) is
begin
if N is a leaf then
return the estimated score of this leaf
else
Let N1, N2, .., Nm be the successors of N;
if N is a Min node then
return min{MINIMAX(N1), .., MINIMAX(Nm)}
else
return max{MINIMAX(N1), .., MINIMAX(Nm)}
end MINIMAX;
Alpha-Beta Pruning
Alpha-beta pruning is a technique that enables the MiniMax
algorithm to ignore branches that will not contribute further to
the outcome of the search. It is a type of Branch-and-Bound
search technique, like the A* with pruning was. Again, this
technique is best demonstrated by an example and then we
shall look at the algorithm more closely.
ab Pruning
Max 4
Min 4
4 10 5 3 9 17 7 2 1
After evaluating left sub-tree, we know Max has move with
value 4. After expanding the first leaf node of the middle
sub-tree, what can we conclude ?
ab Pruning
Max 4
Min 4
4 10 5 3 9 17 7 2 1
The value for Max from the middle sub-tree is less than or
equal to 3. Since Max already has a move with a higher
value, the decision at the root will be unchanged no matter
what the values of the other leaves in the middle sub-tree
are
ab Pruning
Max 4
Min 4
4 10 5 3 9 17 7 2 1
The first leaf of the right sub-tree promises a value of 7 or
lower. So we continue to the second leaf which brings the
value of sub-tree down to 2. At this point Max can stop
looking at this sub-tree because its value will be less than
or equal to 2 and Max has a move with a value of 4
Alpha-Beta Cutoff
function MINIMAX_AB(N, A, B) is
begin
Set Alpha value of N to A and Beta value of N to B;
if N is a leaf then return the estimated score of this leaf
else if N is a MAX node then
for each successor Ni of N loop
Let Val be MINIMAX_AB(Ni, Alpha of N, Beta of N);
Set Alpha value of N to Max(Alpha value of N, Val);
When Alpha value of N > Beta value of N then exit loop;
Return Alpha value of N;
else ……{N is a MIN node}
for each successor Ni of N loop
Let Val be MINIMAX_AB(Ni, Alpha of N, Beta of N);
Set Beta value of N to Min(Beta value of N, Val);
When Beta value of N < Alpha value of N then exit loop;
Return Beta value of N;
end MINIMAX_AB;
Harga awal = - dan = = (3) (4)
A MAX
= - , = 3
3 =-, maka B di
explore lebih lanjut = (5) (4)
5 =3, maka C di
B explore C MIN
3 D 5 E
0 = , maka F di 7 > =5, maka G
explore F G di pruned. H MAX
4
= 3 , =
0 =3, maka I I J M N
di pruned MIN
5 7 8
0 K L 7
a=4
max
2<a
min b=4
1b=∞, maka n
6 > b=4 di explore
max 4 b=∞
3 < a =4 1< a=4 2< a
min 6 a
b=4
A MAX
B C D
E F G H I J
2 3 5 9 0 7 4 2 1 5 6
A MAX
B C D
E F G H I J K
7 6 8 5 2 3 0 -2 6 2 5 8 9 2
Perform MiniMax dan left2right alpha-beta pruning pada tree
di atas
MAX a=3 A 2 < a = 3, maka
D di prunned
MIN 3 B 0 C a = 3, b = D
2 < b = , maka I
di explore
MAX 3 5 F 0 7 6
2 3 5 9 0 7 4 2 1 5 6
- B has = 3 (B will be no larger than 3)
- F is pruned since 5 > b = 3
- A has = 3 (A will be no smaller than 3)
- C is pruned, since 0 < a =3
- D is pruned, since 2 < a = 3
a = (3) 8 A
MAX
0<a=3
C
b = (7) 3 B D b=8 MIN
0 b = , H
5 b = , J
di explore
di explore
a=7 E 8 F 3 G H 6 I J K 9
7 6 8 5 2 3 0 -2 6 2 5 8 9 2
- F is pruned (8 > b = 7)
- B has = 3 (B will be no larger than 3)
- A has = 3 (A will be no smaller than 3)
- C is pruned (0 < a = 3)
- K is b pruned (9 > b = 8)
- A has a = 8 (A will be no smaller than 8)
Problem 1
a=?ẇ max
min
max
-70 76 49 43 -75 -18 3 -25 -68 99 48 -65 90 -22 82 -26
Question : Perform left2right a-b pruning to determine a.
Problem 2
a=1 max
min
max
79 1 -46 -3 55 24 69 74 -12 92 -62 -82 -77 95 51 -74
Question : Perform left2right a-b pruning to determine a.
