Extensive-Form Games

with Imperfect Information

Yiling Chen

September 20, 2010

Perfect Information vs. Imperfect Information

I Perfect Information
I All players know the game structure.

I Each player, when making any decision, is perfectly

informed of all the events that have previously
occurred.

I Perfect Information
I All players know the game structure.

I Each player, when making any decision, may not be

perfectly informed about some (or all) of the events
that have already occurred.
Roadmap

I Define Imperfect-Information Extensive-Form

Game
I Introduce Sequential Equilibrium
”rather a lot of bodies are buried in this
definition”. (Kreps, 1990)
Example 1

L M R
2
(2, 2)
a r a r

(3,1) (0,0) (0,2) (1,1)

Example 2

L R
1

0
L R

0 1
Def. of Imperfect-Information Extensive-Form Games
I An imperfect-information extensive-form game is a tuple
(N, H, P, I, u)
I (N, H, P, u) is a perfect-information extensive-form game
I I = {I1 , I2 , ..., In } is the set of information partitions of
all players
I Ii = {Ii,1 , ...Ii,ki } is the information partition of player i
I Ii,j is an information set of player i
I Action set A(h) = A(h0 ) if h and h0 are in Ii,j , denote as
A(Ii,j )
I P(Ii,j ) be the player who plays at information set Ii,j .
I1 = {{Φ}, {(L, A), (L, B)}}, I2 = {{L}}

1
L R
2
A B (2,1)
1
a b a b

(0,0) (1,2) (1,2) (0,0)

Pure Strategies: Example 3

I S = {S1 , S2 }
I S1 = {(L, a), (L, b), (R, a), (R, b)}
I S2 = {A, B}

1
L R
2
A B (2,1)
1
a b a b

(0,0) (1,2) (1,2) (0,0)

Normal-Form Representation: Example 1
An imperfect-information extensive-form game ⇒ A
normal-form game

1 a r

L M R L 2,2 2,2
2
M 3,1 0,0
(2, 2)
a r a r
R 0,2 1,1
(3,1) (0,0) (0,2) (1,1)

The Nash Equilibrium (both pure and mixed) concept remains

the same for imperfect-information extensive-form games.
Normal-Form Representation: Example 1
An imperfect-information extensive-form game ⇒ A
normal-form game

1 a r

L M R L 2,2 2,2
2
M 3,1 0,0
(2, 2)
a r a r
R 0,2 1,1
(3,1) (0,0) (0,2) (1,1)

The Nash Equilibrium (both pure and mixed) concept remains

the same for imperfect-information extensive-form games.
Normal-Form Games

A normal-form game ⇒ An imperfect-information

extensive-form game

C D
1

C 5, 5 0, 8
C D
2
D 8, 0 1, 1
C D C D

Prisoner’s Dilemma (5,5) (0,8) (8,0) (1,1)

Nash Equilibrium: Example 1

1 a r
L M R L 2,2 2,2
2
M 3,1 0,0
(2, 2)
a r a r
R 0,2 1,1

(3,1) (0,0) (0,2) (1,1)

Suppose we want to generalize the idea of subgame perfect

equilibrium. Consider the equilibrium (L, r). Is it subgame
perfect?
Nash Equilibrium: Example 1

1 a r
L M R L 2,2 2,2
2
M 3,1 0,2
(2, 2)
a r a r
R 0,2 1,1

(3,1) (0,2) (0,2) (1,1)

Suppose we want to generalize the idea of subgame perfect

equilibrium. Consider the equilibrium (L, r). Is it subgame
perfect?
Beliefs
I A belief µ is a function that assigns to every information set a
probability measure on the set of histories in the information
set.
I An assessment in an extensive-form game is a strategy-belief
pair (s, µ).
I The assessment (s, µ) is sequentially rational if for every
player i and every information set Ii,j ∈ Ii we have
E [ui (si , s−i |Ii,j )] ≥ E [ui (si0 , s−i |Ii,j )]
for any si0 6= si .
1 a r

L M R L 2,2 2,2
2
M 3,1 0,2
(2, 2)
a r a r
R 0,2 1,1

(3,1) (0,2) (0,2) (1,1)

Restrictions to beliefs?

I In lieu of the Nash equilibrium concept, we require that

beliefs are derived from equilibrium strategies according to
Bays rule (as if players know each others strategies).
I But, what about beliefs for information sets that are off
the equilibrium path?

1 a r

L M R L 2,2 2,2
2
M 3,1 0,2
(2, 2)
a r a r
R 0,2 1,1

(3,1) (0,2) (0,2) (1,1)

Restrictions to Beliefs?
I We want beliefs for information sets that are off the
equilibrium path to be reasonable. But what is
reasonable?
Consider the NE (L, r) again. Player 2’s information set will not be
reached at the equilibrium, because player 1 will play L with
probability 1. But assume that player 1 plays a completely mixed
strategy, playing L, M, and R with probabilities 1 − , 3 
4 , and 4 .
Then, the belief on player 2’s information set is well defined. Now,
if  → 0, it’s still well defined.
1 a r

L M R L 2,2 2,2
2
M 3,1 0,2
(2, 2)
a r a r
R 0,2 1,1

(3,1) (0,2) (0,2) (1,1)

Consistent Assessment

I An assessment (s, µ) is consistent if there is a sequence

((s n , µn ))∞
n=1 of assessments that converges to (s, µ) and
has the properties that each strategy profile s n is
completely mixed and that each belief system µn is
derived from s n using Bayes rule.
Sequential Equilibrium

I An assessment (s, µ) is a sequential equilibrium of a finite

extensive-form game with perfect recall if it is sequentially
rational and consistent.
I Thm: Every finite extensive-form game with perfect recall
has a sequential equilibrium.
I A sequential equilibrium is a Nash equilibrium.
I With perfect information, a subgame perfect equilibrium
is a sequential equilibrium.
Bayesian Games

Yiling Chen

September 20, 2010

So far

Up to this point, we have assumed that players know all

relevant information about each other. Such games are known
as games with complete information.
Games with Incomplete Information

I Bayesian Games = Games with Incomplete Information

I Incomplete Information: Players have private information
about something relevant to his decision making.
I Incomplete information introduces uncertainty about the
game being played.

I Imperfect Information: Players do not perfectly observe

the actions of other players or forget their own actions.

We will see that Bayesian games can be represented as

extensive-form games with imperfect information.
Example 4: A Modified Prisoner’s Dilemma Game
With probability λ, player 2 has the normal preferences as
before (type I), while with probability (1 − λ), player 2 hates
to rat on his accomplice and pays a psychic penalty equal to 6
years in prison for confessing (type II).

λ 1−λ
C D C D

C 5, 5 0, 8 C 5, 5 0, 2

D 8, 0 1, 1 D 8, 0 1, -5

Type I Type II
Simultaneous-Move Bayesian Games
I A simultaneous-move Bayesian game is (N, A, Θ, F , u)
I N = {1, ..., n} is the set of players
I A = {A1 , A2 , ..., An } is the set of actions
Ai = {Cooperation, Defection}.
I Θ = {Θ1 , Θ2 , ..., Θn } is the set of types. θi ∈ Θi is a
realization of types for player i.
Θ2 = {I, II}.
I F : Θ → [0, 1] is a joint probability distribution, according to
which types of players are drawn
p(θ2 = type I) = λ
I u = {u1 , u2 , ..., un } where ui : A × Θ → R is the utility
function of player i
I Two assumptions
I All possible games have the same number of agents and the
same action spaces for each agent
I Agents have common prior. The different beliefs of agents are
posteriors.
Imperfect-Information Extensive-Form
Representation of Bayesian Games
I Add a player Nature who has a unique strategy of
randomizing in a commonly known way.

Nature

λ 1−λ
1

C D C D
2 2

C D C D C D C D

(5, 5) (0, 8) (8,0) (1, 1) (5, 5) (0, 2) (8, 0) (1, -5)

Strategies in Bayesian Games

I A pure strategy si : Θi → Ai of player i is a mapping from

every type player i could have to the action he would play
if he had that type. Denote the set of pure strategies of
player i as Si .
S1 = {{C}, {D}}
S2 = {{C if type I, C if type II}, {C if type I, D if type
II}, {D if type I, C if type II}, {D if type I, D if type II}}
I A mixed strategy σi : Si → [0, 1] of player i is a
distribution over his pure strategies.
Best Response and Bayesian Nash Equilibrium
We use pure strategies to illustrate the concepts. But they
hold the same for mixed strategies.
I Player i’s ex ante expected utility is
X
Eθ [ui (s(θ), θ)] = p(θi )Eθ−i [ui (s(θ), θ)|θi ]
θi ∈Θi

I Player i’s best responses to s−i (θ−i ) is

BRi = arg max Eθ [ui (si (θi ), s−i (θ−i ), θ)]

si (θi )∈Si
X  
= p(θi ) arg max Eθ−i [ui (si (θi ), s−i (θ−i ), θ)|θi ]
si (θi )∈Si
θi ∈Θi

I A strategy profile si (θi ) is a Bayesian Nash Equilibrium iif

∀i si (θi ) ∈ BRi .
Bayesian Nash Equilibrium: Example 4
I Playing D is a dominant strategy for type I player 2; playing C
is a dominant strategy for type II player 2.
I Player 1’s expected utility by playing C is
λ × 0 + (1 − λ) × 5 = 5 − 5λ.
I Player 1’s expected utility by playing D is
λ × 1 + (1 − λ) × 8 = 8 − 7λ > 5 − 5λ.
I (D, (D if type I, C if type II)) is a BNE of the game.
λ 1−λ
C D C D

C 5, 5 0, 8 C 5, 5 0, 2

D 8, 0 1, 1 D 8, 0 1, -5

Type I Type II
Signaling (Sender-Receiver Games)

I There are two types of workers, bright and dull.

I Before entering the job market a worker can choose to get
an education (i.e. go to college), or enjoy life (i.e. go to
beach).
I The employer can observe the educational level of the
worker but not his type.
I The employer can hire or reject the worker.
Example 5: Signaling

Nature

λ Bright Dull 1−λ

Worker Worker
C B C B
Employer
Employer

H R H R H R H R

(2, 2) (-1, 0) (4,-1) (1, 0) (2, 1) (-1, 0) (4, -2) (1, 0)

Bayesian Extensive Games with Observable Actions

I A Bayesian extensive game with observable actions is

(N, H, P, Θ, p, u)
I (N, H, P) is the same as those in an extensive-form game with
perfect information
I Θ = {Θ1 , Θ2 , ..., Θn } is the set of types. θi ∈ Θi is a
realization of types for player i.
Θ1 = {Bright, Dull}.
I F : Θ → [0, 1] is a joint probability distribution, according to
which types of players are drawn
p(θ1 = Bright) = λ
I u = {u1 , u2 , ..., un } where ui : Z × Θ → R is the utility
function of player i. Z ∈ H is the set of terminal histories.
Best Responses for Example 5
I E.g. If the employer always plays H, then the best
response for the worker is B.
I But how to define best responses for the employer?
I Beliefs on information sets
I Beliefs derived from strategies

Nature

λ Bright Dull 1−λ

Worker Worker
C B C B
Employer
Employer

H R H R H R H R

(2, 2) (-1, 0) (4,-1) (1, 0) (2, 1) (-1, 0) (4, -2) (1, 0)

A Bayesian Nash Equilibrium of Example 3

Nature

λ Bright Dull 1−λ

Worker Worker
C B C B
Employer
Employer
λ 1−λ
H R H R H R H R

(2, 2) (-1, 0) (4,-1) (1, 0) (2, 1) (-1, 0) (4, -2) (1, 0)

p(Bright|Beach) =
p(Bright)σ(Beach|Bright) λ·1
p(Bright)σ(Beach|Bright)+p(Dull)σ(Beach|Dull)
= λ·1+(1−λ)·1
=λ
“Subgame Perfection”

I The previous Bayesian Nash Equilibrium is not “subgame

perfect”. When the information set College is reached,
the employer should choose to hire no matter what belief
he has.
I We need to require sequential rationality even for
off-equilibrium-path information sets.
I Then, beliefs on off-equilibrium-path information sets
matter.
Perfect Bayesian Equilibrium
A strategy-belief pair, (σ, µ) is a perfect Bayesian equilibrium
if
I (Beliefs) At every information set of player i, the player
has beliefs about the node that he is located given that
the information set is reached.
I (Sequential Rationality) At any information set of player i,
the restriction of (σ, µ) to the continuation game must be
a Bayesian Nash equilibrium.
I (On-the-path beliefs) The beliefs for any
on-the-equilibrium-path information set must be derived
from the strategy profile using Bayes’ Rule.
I (Off-the-path beliefs) The beliefs at any
off-the-equilibrium-path information set must be
determined from the strategy profile according to Bayes
Rule whenever possible.
Perfect Bayesian Equilibrium

I Perfect Bayesian equilibrium is a similar concept to

sequential equilibrium, both trying to achieve some sort of
“subgame perfection”.
I Perfect Bayesian equilibrium is defined for all
extensive-form games with imperfect information, not just
for Bayesian extensive games with observable actions.
I Thm: For Bayesian extensive games with observable
actions, every sequential equilibrium is a Perfect Bayesian
equilibrium.
A Perfect Bayesian Equilibrium of Example 3

Nature

λ Bright Dull 1−λ

Worker Worker
C B C B
Employer
Employer
λ β 1−λ 1−β
H R H R H R H R

(2, 2) (-1, 0) (4,-1) (1, 0) (2, 1) (-1, 0) (4, -2) (1, 0)

β ∈ [0, 1]
Summary of Equilibrium Concepts

On-equ-path On-equ-path Off-equ-path Off-equ-path

strategy σon belief µon strategy σoff belief µoff
NE BR N/A N/A N/A
BNE BR given µon Consistent N/A N/A
with σon
SPNE BR N/A BR N/A
PBE BR given µon Consistent BR given µoff Consistent
with σon with σoff
SE BR given µon Consistent BR given µoff Consistent
with σon with σoff
Paper Presentations

I About 20-30 minutes per paper.

I Presentation: a short summary + a critique
I You are asked to come to talk with the teaching staff
before your presentation.
I After the presentation, we will break into discussions
under the assumption that everyone has read the paper.
Reading Papers
When you read papers and write your comments, please think
about the following. (You don’t need to hit on all of them.)
I What is the main contribution of the paper?
I Is it important? Why?
I What is the limitation of the paper?
I What was the main insight in getting the result?
I What assumptions were made? Are they reasonable, limiting,
or overly simplified?
I What applications might arise from the paper?
I How can the results be extended?
I What was unclear to you?
I How does the paper relate to other work that we have seen?
I Can you suggest a two-sentence project idea based on the
ideas in the paper?