Lecture 4 TOPICS IN GAME THEORY

J¨ orgen Weibull

September 18, 2012

• An extensive-form game: captures the dynamics of the interaction in question - who moves when, who observes what, and who cares about what (Bernoulli functions over plays)

• A normal-form game: represents the game structure in a mathematically convenient way - everybody’s strategy sets and payoff functions

1

What is an extensive-form game?

Kuhn (1950, 1953), Selten (1965, 1975), Kreps and Wilson (1982) Γ = hN, A, ψ, P , I , C , p, r, v i where: 1. N = {1, ..., n} the (finite) set of personal players 2. A directed (and connected) tree: (a) A (finite) set A of nodes, with a root, a0 ∈ A (b) A predecessor function:, ψ : A\ {a0} → A (c) From each node a 6= a0, the root can be reached by a finite number of iterations of ψ

(d) a0 precedes a if a0 = ψ (ψ (...ψ (a))). Write a0 < a (e) Aω ⊂ A the terminal nodes, nodes that precedes no node (aω ∈ / ψ (A))

3. T the set of plays τ , ways to go from a0 to Aω

4. P = {P0, P1, ..., Pn} the player partitioning of non-terminal nodes 5. I = ∪i∈N Ii, where each Ii is the information partitioning of Pi ⊂ A into information sets I ∈ Ii. Regularity conditions: (a) Each play intersects every information set at most once (b) All nodes in an info set have the same number of outgoing branches

6. C = {CI : I ∈ I}, each CI being the choice partitioning at I (a) notation c < a

7. p : P0 → [0, 1] the probabilities of “nature’s” random moves 8. r : T → D the result (or outcome ) mapping, assigning material outcomes (money, food,...) to plays 9. v : T → Rn the combined Bernoulli function (6= payoff function) • Φ = hN, A, ψ, P , I , C , pi the game form • Ψ = hN, A, ψ, P , I , C , p, ri the game protocol

ψ. p. A. pH (and can condition on the quality) • Seeing the price. pL. Player 2: a representative buyer • Quality of the car is either H (high) or L (low). I .Example: A Seller-Buyer Game • Player 1: A used-car seller. C . P . observed only by the seller • The seller chooses either a low price. vi? .(not buy) • What is Γ = hN. or a high price. the buyer chooses either B (buy) or N . r.

I . ψ. C . A. P .What is Γ = (N. p. v )? . r.

Example: Is the following an extensive form? B A A B 2 3 B B 3 A 2 A L 1 R .

and denoted Γa (in particular.1. endowed with the same partitionings etc.1 Subgames n o • The follower set F (a) = a0 ∈ A : a ≤ a0 • Subroots are nodes a for which: F (a) ∩ I 6= ∅ ⇒ I ⊂ F (a) . Γa0 = Γ is a subgame) . • A subgame of Γ is the tree starting at a subroot a.

Each pure strategy specifies exactly one choice at each information set “belonging to” the player in question • Pure-strategy profiles s = (s1. realization probabilities and payoff functions 2. .. sn) ∈ S = ×Si • Realization probabilities for plays τ ∈ T : ρ (τ .. s) .2 Strategies.1 Pure strategies • Pure strategies si ∈ Si..

• Mixed-strategy profiles x = (x1.• Payoff functions π i : S → R πi (s) = τ ∈T X ρ (τ .. xn) ∈ X = ¤ (S ) = ×i∆ (Si) . . s) vi (τ ) • Recall that a pure strategy thus is more than what people usually think. As if each player randomizes before starting to play.. 2...2 Mixed strategies • Mixed strategies xi ∈ Xi = ∆ (Si)..

• Realization probabilities: ρ ˜ (τ . x) = s∈S Xh Πj ∈N xj sj ³ ´i ρ (τ . s) • Payoff functions π ˜ i (x) = τ ∈T X ρ ˜ (τ . x) vi (τ ) .

3) (2.2) (0.0) C F 2 A E 1 Game 4 ( π ˜ 1 (x) = x11 + 2x12x21 π ˜ 2 (x) = 3x11 + 2x12x21 .Example: (1.

yiI ∈ YiI = ∆ (CI ) • Behavior strategies: functions yi that assign local strategies to information sets I ∈ Ii.as if players randomize as the play proceeds • Behavior strategy profiles: y ∈ Y = ×i∈N Yi .2. yi ∈ Yi = ×I ∈Ii YiI .3 Behavior strategies • Local strategies: statistically independent randomizations over choice sets.

y ) vi (τ ) • In the preceding example: Y = X because each player has only one info set Definition 2.• Realization probabilities: ρ ˆ (τ .2 Path of strategy profile = the set of plays assigned positive probabilities . y ) = the product of all choice probabilities along τ • Payoff functions π ˆ i (y ) = τ ∈T X ρ ˆ (τ .1 Outcome of strategy profile = induced probability distribution over plays (or end-nodes) Definition 2.

3 Perfect recall and Kuhn’s theorem • Mixed strategies: “global randomizations” performed at the beginning of the play of the game • Behavior strategies: “local randomizations” performed during the course of play of the game • Equivalence in terms of realization probabilities? .

1 (Kuhn 1950. choice c ∈ CI and nodes a. . J ∈ Ii.Definition 3.1953) An extensive form Φ has perfect recall if c < a ⇔ c < a0 for each player i ∈ N . a0 ∈ J . pair of information sets I.

.Bernoulli values and payoffs are irrelevant for the definition of perfect recall Informally: Theorem 3. then perfect recall .All perfect-information games have perfect recall . ∃ a realization-equivalent behavior strategy.1 (“Kuhn’s Theorem”) If Φ has perfect recall.• Note: . for each mixed strategy. then.If each player has only one information set.

. where Wi is the ³ ³ strategies: ´ ³ wi´´ 1 .2 A (behavior-strategy) mixture.To state this more exactly. y k ∈ Y .. is a finite-support randomization over the player’s behavior ∈ Wi. the mixture wi that assigns unit probability to yi. Definition 3... w y k set of probability vectors wi = wi yi for some k ∈ N and i i 1. yi i i • Every behavior strategy yi ∈ Yi can be viewed as a (degenerate) behavior-strategy mixture... wi. . . . • Every mixed strategy xi ∈ Xi can be viewed as the mixture wi that h assigns probability xih ∈ [0. and prove it: Consider a player i in a finite extensive form Φ. 1] to the (degenerate) behavior strategy yi that assigns unit probability to the choices made under pure strategy h ∈ Si.

Selten 1975) Consider a player i in a finite extensive form Φ with perfect recall. . Theorem 3.2 (Kuhn 1950. w−i are identical for all mixture profiles w00 ∈ W = ×n j =1Wj .3 Mixtures wi. w−i and wi. For each behavior-strategy mixture 0 ∈ W that assigns wi ∈ Wi there exists a realization-equivalent mixture wi i unit probability to a behavior strategy yi ∈ Yi. wi i³ ´ ³ ´ 0 00 00 ization probabilities under the profiles wi.0 ∈ W are realization equivalent if the realDefinition 3.

. Note that conditional probabilities over the nodes in any of i0s information sets I do not depend on i’s own strategy • The “behavior” induced at information sets by a mixed-strategy profile • Equivalence in terms of realization probabilities • We henceforth will assume perfect recall.Proof sketch: 1. Consider those of i’s information sets I that ³ ´ are possible under wi in 00 00 the sense that I is on the path of wi. w− i for some w ∈ W 2.

The behavior-strategy normal form 4. The pure-strategy normal form 2. The reduced normal form . The quasi-reduced normal form 5.4 The five NF-games associated with an EF-form game Five normal forms associated with any given extensive form: 1. The mixed-strategy normal form 3.

X. π • The mixed-strategy normal-form: G ˜ i where X = ×i∈N Xi is the set of mixed-strategy profiles x = (xi)i∈N where Xi = 4 (Si) π ˜ : X → Rn is the combined payoff function. πi where The strategy set of player i is Si = ×I ∈Ii CI S = ×i∈N Si is the set of pure-strategy strategy profiles s = (si)i∈N π : S → Rn is the combined payoff function.• The pure-strategy normal-form: G = hN. S. π i (s) ∈ R the payoff to player i under s ˜ = hN. π ˜ i (x) ∈ R the payoff to player i under x .

π G ˆ i where Y = ×i∈N Yi is the set of behavior-strategy profiles y = (yi)i∈N where Yi = ×I ∈Ii ∆ (CI ) π ˆ : Y → Rn is the combined payoff function.• The behavior-strategy normal-form representation of Γ is a triplet ˆ = hN. Y. π ˆ i (y ) ∈ R the payoff to player i under y .

s−i) = π ˜ (xi. s−i = π si . s−i) ∀s−i ∈ S−i.1 A player’s two pure strategies. π i if there exists a mixed strategy xi ∈ 4 (Si) such that π (si.2 A pure strategy.00 Definition 4. ³ • Note that one requires that all players should be indifferent • The quasi-reduced normal-form game is the triplet G = hN. S. s0 i. π i where all equivalent pure-strategy pairs have been replaced by a single pure strategy until no further such replacement is possible. si ∈ Si are equivalent if ´ ³ ´ 0 00 π si. s−i ∀s−i ∈ S−i. . si ∈ Si is redundant in a pure-strategy normal form game G = hN. Definition 4. S.

S. . πi where all equivalent and redundant pure strategies have been removed.• The reduced normal-form game is the pure-strategy normal-form game G = hN.

2 D 2. 2 0. 0 .2) (0. 1 2. 2 1. 1 2.1) 2 D A B 1 Pure-strategy normal form: AE AF BE BF C 2.Example: (3. 1 1.0) E (1.2) 1 F C (2. 1 3.

0 . 2 BF 1. 1 2. 1 BE 1.Quasi-reduced (and reduced) normal-form representation: C D A 2. 2 3. 2 0.

5 Solution concepts • Now we are in a position to define and analyze different solution concepts for extensive-form games. subgame perfect equilibrium. and extensive-form perfect equilibrium . • Focus on behavior strategies in finite EF games with perfect recall (recall “Kuhn’s Theorem”) • Solution concepts: Nash equilibrium. sequential equilibrium. perfect Bayesian equilibrium.

5. Y. π • Let G ˆ ) be the behavior-strategy NF representation of Γ ˆ always exist? Q1: Do NE in G ˆ need not • Mathematical difficulty: the best-reply correspondence in G be convex-valued .1 Nash equilibrium ˆ = (N.

1 ∃ NE of G ˆ ). π Proof : Consider mixed-strategy NF G ˜ ) and use Kuhn’s Theorem! . X. ˜ = (N. π Proposition 5.ˆ = (N. Y.

Definition: I ∈ I is on the path of y ∈ Y if ρ ˆ (I. y ) ∀a ∈ I . the conditional probabilities over the node in I are Pr (a | y ) = ρ ˆ (a. y ) ρ ˆ (I.Q2: How find NE directly in the extensive form? 1. and let ρ ˆ (I. y ) 2. For any node a. y ) > 0 3. let ρ ˆ (a. By Bayes’ rule. and behavior-strategy profile y . information set I . y ) be the probability that node a is reached when y is played. y ) = a∈I X ρ ˆ (a. Consider I on the path of y .

Definition 5. yi ∈ Yi is a best reply to y at all information sets I ∈ Ii on its path. y Pr (a | y ) · π ia yi −i ³ ´ 0 ∈Y ∀yi i . a∈I X 0. A behavior ∗ ∈ Y is a best reply to y at I if: strategy yi i a∈I X ∗.Definition 5. y−i) is the conditional payoff-function for player i when play starts at node a ∈ I .1 Suppose that I ∈ Ii is on the path of y ∈ Y . y ) ≥ Pr (a | y ) · π ia (yi −i where π ˆ ia (·.2 A behavior-strategy profile y ∈ Y is sequentially rational on its own path in Γ if. for all players i ∈ N .

Proposition 5. of G Proof sketch: ˆ . Then it prescribes a suboptimal 1. Suppose that y is not a NE of G move somewhere on its own path 2. 1984) A behavior-strategy profile y is a NE ˆ iff it is sequentially rational on its own path in Γ. Suppose that y does not prescribe a best reply to itself at some info set on its path .2 (van Damme.

. For generic payoffs.5. this SPE is unique.3 Every finite EF games of perfect information has at least one SPE in pure strategies.2 Subgame perfection Definition 5.3 (Selten. • Proof sketch in class: use Kuhn’s algorithm. 1965) A behavior strategy profile y is a subgameperfect equilibrium (SPE) of Γ if its restriction y a to each subgame Γa is ˆ a. a NE of G • In simultaneous-move games: SPE=NE • In games of perfect information: Proposition 5.

4 Every finite EF game with perfect recall has at least one SPE. • Proof sketch in class: use a generalized version of Kuhn’s algorithm. .• By a slight generalization of Kuhn’s algorithm: Proposition 5.

3) b A (2.1 Consider a battle-of-the-sexes game in which player 1 has an outside option (go to a caf´ e with a friend): (3.0) a (1.0) b 1 (0.v) 2 L R B 1 Find all subgame perfect equilibria! (Does the value v ∈ R matter?) .1) a (0.Example 5.

Example 5. In this extensive-form game.2 Add a payoff-equivalent move to the entry-deterrence game. F ) becomes subgame perfect (although still implausible): 0 0 F Y 2 2 2 0 0 F Y 2 2 1 3 A E1 E2 1 .• However. SPE is sensitive to details of the EF form. s = (A.

.3 “Selten’s horse” 0 0 0 L R 3 3 2 2 0 0 1 L R 4 4 0 1 1 1 R L 2 L R 1 • y = (L.• Subgame perfection . R) is a NE and hence a SPE. sequential rationality at all singleton-information sets Example 5. R.

• But 2’s move is not a best reply to y • In all solution concepts: a strategy is not only a “contingent plan” for the player in question. but also others’ expectation about the player’s moves. .

1] such that a∈I X μ (a) = 1 ∀I ∈ I Definition 5.4 A belief system is a function μ : AÂAω → [0.3 Sequential equilibrium • Refine SPE by generalizing the stochastic dynamic programming approach from 1 decision-maker to n decision-makers • Kreps and Wilson (1982) Definition 5.5 An belief system μ is consistent with a behavior-strategy t → y such profile y if ∃ sequence of interior behavior-strategy pro fi les y ³ ´ t that Pr · | y → μ (·).5. .

Proposition 5. y μ (a) π ia yi −i ³ ´ 0 ∈Y ∀yi i Definition 5. .5 Every sequential equilibrium is subgame perfect. y ) > 0 ⇒ μ agrees with Pr (· | y ) on I Definition 5.6 A behavior-strategy profile y is sequentially rational under a belief system μ if for every player i and each information set I ∈ Ii: a∈I X μ (a) πia (y ) ≥ a∈I X 0.7 A behavior-strategy profile y is a sequential equilibrium (SE) if y is sequentially rational under some belief system μ that is consistent with y .• ρ ˆ (I.

• Reconsider: (1) The above battle-of-sexes with an outside option (2) The above entry-deterrence game with added move (3) Selten’s “horse” (see next slide) .

since s0 2=L . is not sequentially rational.4 Reconsider “Selten’s horse” 0 0 0 L R 3 2 2 0 0 1 L 3 R 4 4 0 1 1 1 R L 2 L R 1 We first note that the arguably unsatisfactory subgame-perfect equilibrium s = (L. R) is not a sequential equilibrium.Example 5. R. s2 = R. The reason is that player 2’s (pure) behavior strategy.

Under such a strategy profile. Her strategy L is sequentially rational iff μ (a) ≥ 1/3. and in that case. against s (yielding a payoff of 4 instead of 1) Next. then player 2’s strategy in y . if she would play R with a probability > 1/4. any strategy for her is sequentially rational. R. 3’s information set is not on the path. sequential equilibrium requires that y3L ≥ 3/4. .is a better reply than R. would not be sequentially rational. and any conditional probabilities μ (a) and μ (b) (both non-negative and summing to 1) that player 3 may attach to her 2 nodes are part of a consistent belief system. and she is indifferent between her strategies L and R iff μ (a) = 1/3. at 2’s node. R. y3). we note that all sequential equilibria are of the form y = (R. where y3 assigns probability ≥ 3/4 to L. Hence. However.

on each information set on y ’s path. Definition 5. but no consensus among game theorists .4 Perfect Bayesian equilibrium • Relaxing the consistency requirement in sequential equilibrium: Definition 5.8 A belief system μ is weakly consistent with a behaviorstrategy profile y if μ agrees with the conditional probability distribution Pr (· | y ). induced by y .9 A behavior-strategy profile y is a ( weak) perfect Bayesian equilibrium (PBE) if there exists a weakly consistent belief system μ under which y is sequentially rational.5. • Clearly SE ⇒ PBE ⇒ NE • There are more restrictive definitions of PBE.

there exists a realization-equivalent SE y ∗ in Γ.1 Let G be a finite game with mixed strategy extension G ˜ and every EF game Γ with G as its For every proper equilibrium x∗ in G NF. ∃ a sequence of εt-proper profiles xt ∈ int [¤ (S )] with εt → 0 and xt → x∗ . ˜ . G 1. Γ and x∗ be as stated Proof sketch: Let G. Proposition 6. Kohlberg and Mertens (1986)]: ˜.6 Properness implies sequentiality • The normal-form refinement properness has a beautiful implication for extensive-form analysis [van Damme (1983).

For each t ∈ N. But this leads to a contradiction.2. Since xt → x∗. y t → y ∗ ∈ Y . Then μ∗ = limt→∞ μt is a belief system consistent with y ∗ 6. Suppose that y ∗ is not sequentially rational under μ∗: ∃ player i. y−i > a∈I ³ 7. each info set in Γ is on the path of y t 4. 0 ∈ Y such that y 0 6= y ∗ and information set I ∈ Ii and yi i iI iI a∈I X μ∗ (a) π ˆ ´ X 0 ∗ μ∗ (a) π ˆ ia (y ∗) ia yi. Sufficient to verify that y ∗ is a SE! ³ ´ t t 5. let μ = η · | y . Since xt ∈ int [¤ (S )]. For each xt ∃ a realization-equivalent behavior-strategy y t in Γ 3. .

3) b A (2.v) 2 L R B 1 .1) a (0.7 Forward induction • Discussion in class of the outside-option game in the light of ”forward induction” reasoning (see Kohlberg and Mertens. 1986): (3.0) a (1.0) b 1 (0.

THE END .