Game Playing

Chapter 8
 Outline

Overview
Minimax search
Adding alpha-beta cutoffs
Additional refinements
Iterative deepening
Specific games

2
 Overview

Old beliefs
Games provided a structured task in which it was very easy to
measure success or failure.

Games did not obviously require large amounts of knowledge,

thought to be solvable by straightforward search.

3
 Overview
Chess
The average branching factor is around 35.

In an average game, each player might make 50 moves.

One would have to examine 35100 positions.

4
 Overview

Improve the generate procedure so that only good

moves are generated.

Improve the test procedure so that the best moves will

be recognized and explored first.

5
 Overview

Improve the generate procedure so that only good

moves are generated.

plausible-moves vs. legal-moves

Improve the test procedure so that the best moves will

be recognized and explored first.
less moves to be evaluated

6
 Overview

It is not usually possible to search until a goal state is

found.

It has to evaluate individual board positions by

estimating how likely they are to lead to a win.
Static evaluation function

Credit assignment problem (Minsky,1963).

7
 Overview

Good plausible-move generator.

Good static evaluation function.

8
 Minimax Search

Depth-first and depth-limited search.

At the player choice, maximize the static evaluation of

the next position.
At the opponent choice, minimize the static evaluation
of the next position.

9
 Minimax Search

A -2 Maximizing ply
Player
B -6 C -2 D -4
Minimizing ply
E F G H I J K Opponent
9 -6 0 0 -2 -4 -3

Two-ply search

10
 Minimax Search

Player(Position, Depth):
for each S SUCCESSORS(Position) do
RESULT = Opponent(S, Depth + 1)

NEW-V
ALUE = V
ALUE(RESUL
T)

if NEW-VALUE > MAX-SCORE, then

MAX-SCORE = NEW-V
ALUE
BEST-P
A
TH = P
A
TH(RESUL
T) + S
return
V
ALUE = MAX-SCORE
P
A
TH = BEST-P
A
TH
11
 Minimax Search

Opponent(Position, Depth):
for each S SUCCESSORS(Position) do
RESULT = Player(S, Depth + 1)

NEW-V
ALUE = V
ALUE(RESUL
T)

if NEW-VALUE < MIN-SCORE, then

MIN-SCORE = NEW-V
ALUE
BEST-P
A
TH = P
A
TH(RESUL
T) + S
return
V
ALUE = MIN-SCORE
P
A
TH = BEST-P
A
TH
12
 Minimax Search

Any-Player(Position, Depth):
for each S SUCCESSORS(Position) do
RESULT = Any-Player(S, Depth + 1)

ALUE = - V
NEW-V ALUE(RESUL
T)

if NEW-VALUE > BEST-SCORE, then

BEST-SCORE = NEW-V
ALUE
BEST-P
A
TH = P
A
TH(RESUL
T) + S
return
V
ALUE = BEST-SCORE
P
A
TH = BEST-P
A
TH
13
 Minimax Search

MINIMAX(Position, Depth, Player):

MOVE-GEN(Position, Player).

ST
A
TIC(Position, Player).

DEEP-ENOUGH(Position, Depth)

14
 Minimax Search
1. if DEEP-ENOUGH(Position, Depth), then return:

V
ALUE = STA
TIC(Position, Player)
P
A
TH = nil

2. SUCCESSORS = MOVE-GEN(Position, Player)

3. if SUCCESSORS is empty, then do as in Step 1

15
 Minimax Search
4. if SUCCESSORS is not empty:
RESUL
T-SUCC = MINIMAX(SUCC, Depth+1, Opp(Player))
NEW-V
ALUE = - V
ALUE(RESUL
T-SUCC)
if NEW-V
ALUE > BEST-SCORE, then:
BEST-SCORE = NEW-V
ALUE
BEST-P
A
TH = P
A
TH(RESUL
T-SUCC) + SUCC

5. Return:
V
ALUE = BEST-SCORE
P
A
TH = BEST-P
A
TH

16
 AddingAlpha-Beta Cutoffs

Depth-first and depth-limited search.

branch-and-bound

At the player choice, maximize the static evaluation of

the next position.
> a threshold

At the opponent choice, minimize the static evaluation

of the next position.
< b threshold

17
 AddingAlpha-Beta Cutoffs

A
Maximizing ply
>3
Player
B 3 C
Minimizing ply
D E F G Opponent
3 5 -5

Alpha cutoffs

18
 AddingAlpha-Beta Cutoffs
A
Maximizing ply
>3
Player
B C
<5 Minimizing ply
Opponent
D E F G H
3 5 Maximizing ply
Player
I J M N
5 7 Minimizing ply
Alpha and Beta Opponent
cutoffs K L
0

19
 AddingAlpha-Beta Cutoffs

Opponent
b
Player
a b a

Opponent
a b

Player
Alpha and Beta
cutoffs

20
Player(Position, Depth, a, b):
for each S SUCCESSORS(Position) do
RESULT = Opponent(S, Depth + 1, a, b)

NEW-V
ALUE = V
ALUE(RESUL
T)

if NEW-VALUE a, then
a = NEW-V
ALUE
BEST-P
A
TH = P
A
TH(RESUL
T) + S

if a b then return
ALUE = a
V
P
A
TH = BEST-P
A
TH
return
ALUE = a
V
P
A
TH = BEST-P
A
TH
21
Opponent(Position, Depth, a, b):
for each S SUCCESSORS(Position) do
RESULT = Player(S, Depth + 1, a, b)

NEW-V
ALUE = V
ALUE(RESUL
T)

if NEW-VALUE b, then
b = NEW-V
ALUE
BEST-P
A
TH = P
A
TH(RESUL
T) + S

if b a then return
ALUE = b
V
P
A
TH = BEST-P
A
TH
return
ALUE = b
V
P
A
TH = BEST-P
A
TH
22
Any-Player(Position, Depth, a, b):
for each S SUCCESSORS(Position) do
RESULT = Any-Player(S, Depth + 1, -b, -a)

ALUE = - V
NEW-V ALUE(RESUL
T)

if NEW-VALUE a, then
a = NEW-V
ALUE
BEST-P
A
TH = P
A
TH(RESUL
T) + S

if a b then return
ALUE = a
V
P
A
TH = BEST-P
A
TH
return
ALUE = a
V
P
A
TH = BEST-P
A
TH
23
 AddingAlpha-Beta Cutoffs

MINIMAX-A-B(Position, Depth, Player, UseTd, PassTd):

UseTd: checked for cutoffs.

PassTd: current best value

24
 AddingAlpha-Beta Cutoffs
1. if DEEP-ENOUGH(Position, Depth), then return:

V
ALUE = STA
TIC(Position, Player)
P
A
TH = nil

2. SUCCESSORS = MOVE-GEN(Position, Player)

3. if SUCCESSORS is empty, then do as in Step 1

25
 AddingAlpha-Beta Cutoffs
4. if SUCCESSORS is not empty:
RESUL
T-SUCC = MINIMAX-A-B(SUCC, Depth + 1,
Opp(Player), - PassTd, - UseTd)
NEW-V
ALUE = - V
ALUE(RESUL
T-SUCC)
if NEW-V
ALUE > PassTd, then:
PassTd = NEW-V
ALUE
BEST-P
A
TH = P
A
TH(RESUL
T-SUCC) + SUCC

if PassTd UseTd, then return:

V
ALUE = PassTd
P
A
TH = BEST-P
A
TH
5. Return:
V
ALUE = PassTd
P
A
TH = BEST-P
A
TH
26
 Additional Refinements

Futility cutoffs
Waiting for quiescence
Secondary search
Using book moves
Not assuming opponents optimal move

27
 Iterative Deepening

Iteration 1

Iteration 2

Iteration 3
28
 Iterative Deepening

Search can be aborted at any time and the best move

of the previous iteration is chosen.
Previous iterations can provide invaluable move-
ordering constraints.

29
 Iterative Deepening

Can be adapted for single-agent search.

Can be used to combine the best aspects of depth-first
search and breadth-first search.

30
 Iterative Deepening
Depth-First Iterative Deepening (DFID)

1. Set SEARCH-DEPTH = 1

2. Conduct depth-first search to a depth of SEARCH-

DEPTH. If a solution path is found, then return it.

3. Increment SEARCH-DEPTH by 1 and go to step 2.

31
 Iterative Deepening
Iterative-Deepening-A* (IDA*)

1. Set THRESHOLD = heuristic evaluation of the start state

2. Conduct depth-first search, pruning any branch when its

total cost exceeds THRESHOLD. If a solution path is
found, then return it.

3. Increment THRESHOLD by the minimum amount it was

exceeded and go to step 2.

32
 Iterative Deepening

Is the process wasteful?

33
 Homework

Presentations
Specific games: Chess State of theArt

Exercises 1-7, 9 (Chapter 12)

34