Outline for today

Stat155
Game Theory Zero sum games
Lecture 6: Solving two player zero-sum games Recall: payoff matrices, mixed strategies, safety strategies, Von
Neumann’s minimax theorem
Solving two player zero-sum games
Peter Bartlett Saddle points
Dominated pure strategies
Solving 2 × 2 games
Equalizing strategies
September 13, 2016

1 / 25 2 / 25

Recall: Two-player zero-sum games Recall: Two-player zero-sum games

Definitions Definitions
Player I has m actions, 1, 2, . . . , m. A mixed strategy is a probability distribution over actions.
Player II has n actions, 1, 2, . . . , n. A mixed strategy for Player I is a vector
The payoff matrix A ∈ Rm×n
represents the payoff to Player I:  
  x1 ( )
a11 a12 · · · a1n  x2  Xm
 
 a21 a22 · · · a2n  x =  .  ∈ ∆m := x ∈ Rm : xi ≥ 0, xi = 1 .
   .. 
A= . . ..  i=1
 .. . . .  xm
am1 am2 · · · amn

If Player I chooses i and Player II chooses j, the payoff to Player I is A mixed strategy for Player II is a vector y ∈ ∆n .
aij and the payoff to Player II is −aij . A pure strategy is a mixed strategy where one entry is 1 and the
The sum of the payoff to Player I and the payoff to Player II is 0. others 0. (This is a canonical basis vector ei .)

3 / 25 4 / 25
Recall: Two-player zero-sum games Recall: Two-player zero-sum games

A safety strategy for Player I is an x ∗ ∈ ∆m that satisfies

min x ∗ > Ay = max min x > Ay .

The expected payoff to Player I when Player I plays mixed strategy y ∈∆n x∈∆m y ∈∆n
x ∈ ∆m and Player II plays mixed strategy y ∈ ∆n is
m X
X n This mixed strategy maximizes the worst case expected gain for
EI ∼x EJ∼y aIJ = xi aij yj Player I.
i=1 j=1
Similarly, a safety strategy for Player II is a y ∗ ∈ ∆n that satisfies
>
= x Ay .
max x > Ay ∗ = min max x > Ay .
x∈∆m y ∈∆n x∈∆m

This mixed strategy minimizes the worst case expected loss for
Player II.

5 / 25 6 / 25

Recall: Two-player zero-sum games Outline

Von Neumann’s Minimax Theorem

For any two-person zero-sum game with payoff matrix A ∈ Rm×n ,

max min x > Ay = min max x > Ay . Zero sum games

x∈∆m y ∈∆n y ∈∆n x∈∆m
Recall: payoff matrices, mixed strategies, safety strategies, Von
Neumann’s minimax theorem
Solving two player zero-sum games
We call the optimal expected payoff the value of the game, Saddle points
Dominated pure strategies
V := max min x > Ay = min max x > Ay . Solving 2 × 2 games
x∈∆m y ∈∆n y ∈∆n x∈∆m
Equalizing strategies

LHS: Player I plays a safety strategy.

RHS: Player II plays a safety strategy.
Safety strategies are optimal strategies.

7 / 25 8 / 25
Saddle points Saddle points

Example Definition
  A pair (i ∗ , j ∗ ) ∈ {1, . . . , m} × {1, . . . , n} is a saddle point for a payoff
−1 1 5 matrix A ∈ Rm×n if
A =  5 3 4
6 2 1 max aij ∗ = ai ∗ j ∗ = min ai ∗ j .
i j

Suppose Player I plays 2 and Player II plays 2.

If Player I plays i ∗ and Player II plays j ∗ , neither player has an
The payoff is a22 = 3.
incentive to change.
Should either player change their strategy?
Think of these as locally optimal strategies for the players.
This pair of pure strategies, (e2 , e2 ), is called a saddle point (or pure
We’ll see that they are also globally optimal strategies.
Nash equilibrium).

9 / 25 10 / 25

Saddle points: Example Saddle points

1 2 3 4 5 6 7 row min
1 -1 1 2 4 8 -2 7 -1
2 5 3 -4 7 3 1 3 1
3 6 2 -5 2 8 -1 2 1
4 3 1 3 -5 3 4 7 1 Theorem
5 1 2 3 5 4 -6 7 1 If (i ∗ , j ∗ ) is a saddle point for a payoff matrix A ∈ Rm×n , then
6 4 8 3 7 5 8 4 3 ei ∗ is an optimal strategy for Player I,
7 3 1 3 -5 3 4 7 1
ej ∗ is an optimal strategy for Player II, and
column max 6 8 3 7 8 8 7 the value of the game is ai ∗ j ∗ .

Is there a saddle point?

Suppose Player I plays 6 and Player II plays 3.
Neither player should change their strategy: (e6 , e3 ), is a saddle point.
Saddle point is a row/column pair where the row min is equal to the
column max. (Need max of row mins = min of column maxes.)
11 / 25 12 / 25
Saddle points Saddle points
Theorem
Saddle point (i ∗ , j ∗ ) (maxi aij ∗ = ai ∗ j ∗ = minj ai ∗ j ) implies ei ∗ , ej ∗ optimal.

Proof
We have seen that we should always prefer to play last but with a
saddle point, the opposite inequality is also true: So if we find a saddle point, the game is easy to solve.
min max x > Ay ≥ max min x > Ay Another way to simplify a two-player zero-sum game is by removing
y ∈∆n x∈∆m x∈∆m y ∈∆n
dominated rows or columns.
≥ min ei>∗ Ay (∗)
y ∈∆n

= ei>∗ Aej ∗ = max x > Aej ∗ (∗)

x∈∆m
>
≥ min max x Ay .
y ∈∆n x∈∆m
The inequalities are all equalities: min max x > Ay = max min x > Ay .
y x x y
This proves von Neumann’s minimax theorem for this case.
And (∗) shows ei ∗ , ej ∗ optimal.
13 / 25 14 / 25

Outline Removing dominated pure strategies

Example: Plus One (a Mendelsohn game)

Each player picks a number in {1, 2, . . . , n}.
If i = j, payoff is $0.
Zero sum games
Recall: payoff matrices, mixed strategies, safety strategies, Von If |i − j| = 1, the higher number wins $1.
Neumann’s minimax theorem If |i − j| ≥ 2, the higher number loses $2.
Solving two player zero-sum games
Saddle points
Dominated pure strategies
Solving 2 × 2 games
Equalizing strategies

15 / 25 16 / 25
Removing dominated pure strategies Removing dominated pure strategies

Example: Plus One Example: Miss-by-one (a Mendelsohn game

Players I and II choose numbers i, j ∈ {1, 2, . . . , 7}.
Player I wins 1 if |i − j| = 1, otherwise the payoff is 0.
Try it!
What is the payoff matrix?
What do you notice about row 5 and row 7?
For Player I, row 5 is always at least as good as row 7.
Player I never needs to play 7:
There is an optimal strategy x ∗ for Player I with x7∗ = 0.
Similarly for columns 5 and 7:
Reduced payoff matrix
Player II never needs to play 5 (can choose y7∗ = 0).
Similarly for row 1 and row 3, and for column 1 and column 3.
Without row 1, columns 3 and 5, we can omit row 4 and column 4.
There is an optimal strategy for each player mixing only 4 numbers.
17 / 25 18 / 25

Outline 2 × 2 games

Finding Player I’s Finding Player II’s

Zero sum games optimal strategy optimal strategy
Recall: payoff matrices, mixed strategies, safety strategies, Von Recall: Pick-a-hand
Neumann’s minimax theorem
Solving two player zero-sum games
Saddle points
Dominated pure strategies
Solving 2 × 2 games
Equalizing strategies

(Karlin and Peres, 2016) (Karlin and Peres, 2016)

19 / 25 20 / 25
2 × 2 games 2 × 2 games

L R Consider the last two cases.

T c d The optimal strategy for Player I is at the intersection of the lines.
B a b This is an equalizing strategy:
Whatever Player II plays, the expected payoff is the same.
Finding Player I’s optimal strategy Writing x1 = Pr(T ), the equations for the lines are:
a>b>c>d a>b>d>c a>c>b>d a>c>d>b a>d>b>c a>d>c>b

a a a a a a V = b + x1 (d − b),
L L L L L d L
b
cb c c d V = a + x1 (c − a).
R b R R d c
R d b R R
d c db cb
Equating gives
B T B T B T B T B T B T a−b
x1 = .
These represent all games (swap L ↔ R, T ↔ B, or both). a−b+d −c
Saddle points? First four games.
What happens if c → d?
Dominated rows? First, second, third.
Then x1 → 1; the game approaches the fourth case.
Dominated columns? First, third, fourth.
21 / 25 22 / 25

2 × 2 games Solving 2 × 2 games

How to solve a 2 × 2 game
1 Check for a saddle point.
(Is the max of row mins = min of column maxes?) Examples
2 If there are no saddle points, find equalizing strategies.  
0 2
1 3
Equalizing strategies satisfy (in standard notation):
 
3 2
x1 a11 + (1 − x1 )a21 = x1 a12 + (1 − x1 )a22 , 0 1
y1 a11 + (1 − y1 )a12 = y1 a21 + (1 − y1 )a22 .  
3 1
Solving gives 0 2
a21 − a22
x1 = ,
a21 − a22 + a12 − a11
a12 − a22
y1 = .
a12 − a22 + a21 − a11
23 / 25 24 / 25
Outline

Zero sum games

Recall: payoff matrices, mixed strategies, safety strategies, Von
Neumann’s minimax theorem
Solving two player zero-sum games
Saddle points
Dominated pure strategies
Solving 2 × 2 games
Equalizing strategies

25 / 25