JAISCR, 2015, Vol. 5, No. 2, pp.

83 – 95
10.1515/jaiscr-2015-0021

EVOLUTIONARY STABLE STRATEGIES IN NETWORKED

GAMES: THE INFLUENCE OF TOPOLOGY

Dharshana Kasthurirathna, Mahendra Piraveenan, Shahadat Uddin

Centre for Complex Systems Research, Faculty of Engineering and IT

The University of Sydney, NSW 2006, Australia
Email: dkas2394@uni.sydney.edu.au, mahendrarajah.piraveenan@sydney.edu.au,
shahadat.uddin@sydney.edu.au

Abstract

Evolutionary game theory is used to model the evolution of competing strategies in a

population of players. Evolutionary stability of a strategy is a dynamic equilibrium, in
which any competing mutated strategy would be wiped out from a population. If a strategy
is weak evolutionarily stable, the competing strategy may manage to survive within the
network. Understanding the network-related factors that affect the evolutionary stability
of a strategy would be critical in making accurate predictions about the behaviour of a
strategy in a real-world strategic decision making environment. In this work, we evaluate
the effect of network topology on the evolutionary stability of a strategy. We focus on two
well-known strategies known as the Zero-determinant strategy and the Pavlov strategy.
Zero-determinant strategies have been shown to be evolutionarily unstable in a well-mixed
population of players. We identify that the Zero-determinant strategy may survive, and
may even dominate in a population of players connected through a non-homogeneous
network. We introduce the concept of ‘topological stability’ to denote this phenomenon.
We argue that not only the network topology, but also the evolutionary process applied and
the initial distribution of strategies are critical in determining the evolutionary stability of
strategies. Further, we observe that topological stability could affect other well-known
strategies as well, such as the general cooperator strategy and the cooperator strategy.
Our observations suggest that the variation of evolutionary stability due to topological
stability of strategies may be more prevalent in the social context of strategic evolution,
in comparison to the biological context.

1 Introduction strategy is an evolutionarily stable strategy (ESS),

Game theory is the science of strategic decision
making among autonomous players [1]. Game the-
ory has its roots in micro-economics and has later
been adopted in myriad fields of study, such as bi-
ology, psychology and computer science [2]. Evo-
lutionary game theory is the branch of study that
has resulted from the adoption of game theory into
evolutionary biology [3]. It is used to study how
a particular strategy or a group of strategies would
evolve over time in a population of players. If a
once it is adopted by a population of players, any
mutated strategy would not be able to invade it
[4]. Evolutionary stability of a strategy could fur-
ther be divided into two sub categories. strong ESS
and weak ESS (also called asymptotic stable strat-
egy and stable strategy [5]). If a strategy is in a
weak evolutionarily stable state, the invading strat-
egy does not completely die-out but its population
does not increase [5].

Brought to you by | University of Sydney Library

Authenticated
Download Date | 4/19/17 6:19 AM
84
Meanwhile, complex networks science is in-
creasingly used to model and analyse human in-
teractions [6, 7]. Many societies where humans or
other sentient agents interact display heterogeneous
network topologies, in which some agents / people
are very well connected while others are sparsely
connected [7, 8]. In such societies or multi-agent
networks [9], the topology of the network of in-
teractions can be quantified by a range of metrics,
including the clustering coefficient, average path
length, global and local assortativity [10, 11, 12],
topological mutual information [13, 14], various
static and dynamic centrality measures [15] and so
forth. Moreover, many of these social systems dis-
play the so-called scale-free property, which is char-
acterised by maximum heterogeneity in terms of
number of connections, and the small-world prop-
erty, where the characteristic path length is compar-
atively small and the clustering comparatively high.
Therefore it is clear that the topology of any society
or agent network must be considered in determining
the dynamics of a system, including how individual
agents make decisions in such systems.
In this work, we study how network topology
affects the evolutionary stability of strategies. We
use a class of strategies known as ‘memory-one
strategies’ in prisoner’s dilemma (PD) game to eval-
uate the effect of network topology on evolution-
ary stability. Iterated prisoner’s dilemma game has
widely been used to model -the strategic decision
making of self-interested opponents [16, 17, 18].
In memory-one strategies, each player would base
his action on a probability derived based on the
previous interaction with the same opponent [19,
20]. In this work, we particularly focus on a sub-
class of memory-one strategies known as ‘Zero-
Determinant(ZD) strategies’, along with other well-
known memory-one strategies. Zero-determinant
strategies have been demonstrated to be extortion-
ate strategies, meaning that they have the ability to
unilaterally set the payoff of the opponent [21]. In-
tuitively, this would suggest that Zero-Determinant
strategies have the potential to be evolutionarily sta-
ble against any competing strategy. However, Zero-
determinant strategies have later been shown to be
evolutionarily unstable in a well-mixed population
of players [18]. In this work, we test whether net-
work topology affects the evolutionary stability of
Zero-determinant strategies in a non-homogeneous
network of players.
Dharshana Kasthurirathna, Mahendra Piraveenan, Shahadat Uddin
To test the effect of network topology, we use
two well known network classes, known as the
scale-free networks [7] and well-mixed networks.
Also, we use two evolutionary processes to evolve
the populations. First, we test the effect of topology
using the death-birth Moran process [22], which
is an evolutionary process used to model the evo-
lution of players over time, particularly in biolog-
ical systems. Then we use a stochastic strategy
adoption process that would update the strategy of
a randomly selected node, by comparing it with a
selected neighbour’s strategy. The same process
has been used as an evolutionary process by San-
tos et al. [23], with the pure-strategy PD game. We
will refer to this evolutionary process as the ‘strat-
egy adoption process’ in the rest of the paper. We
compare and contrast the evolutionary outcomes
of these two processes when players are placed in
both well-mixed networks and scale-free networks.
Based on the results of these observations, we argue
that the evolutionarily unstable strategies in a well-
mixed population may survive and even dominate
in a heterogeneous network of players. We show
that the topology of the interactions of the players,
the evolutionary update process as well as the ini-
tial topological distribution of players are signifi-
cant determining the overall evolutionary stability
of a strategy. When the players are distributed in
a homogeneous network however, the evolutionary
process used would not have a significant effect the
evolutionary stability of a strategy. We further test
the effect of topology on the evolutionary stability
by varying network assortativity [10], which is a
measure of the similarity of mixing of nodes in a
network. We argue that when the network becomes
more heterogeneous, network topology would have
a more significant effect on the evolutionary stabil-
ity of strategies. We call this topologically influ-
enced evolutionary stability of strategies as ‘topo-
logical stability’.
Understanding the topological effect on the evo-
lutionary stability of a strategy would help us to
make better predictions about the evolutionary sta-
bility of a strategy in a real-world environment.
Even though a strategy may be theoretically sta-
ble or not, its actual evolutionary behaviour may
depend on the topology of the interconnections in
the population and the evolutionary update mech-
anism used. By studying these effects extensively,
the modelling of evolutionary games may be im-
Brought to you by | University of Sydney Library
Authenticated
Download Date | 4/19/17 6:19 AM
EVOLUTIONARY STABLE STRATEGIES IN . . .
proved, by increasing the accuracy of the predic-
tions of the evolutionary stability of strategies.
The rest of the paper is organised as follows.
We begin by providing a background on the the-
oretical aspects of evolutionary game theory and
complex network science, within the scope of this
work. Therein we introduce the memory-one strate-
gies in the iterated prisoner’s dilemma (IPD) game,
as well as a sub-class of memory-one strategies
known as Zero-determinant strategies. Further, we
explain death-birth Moran process and its signifi-
cance in determining the evolutionary stability of
a strategy, comparing it with the strategy adoption
process suggested by Santos et al. [23]. Next, we
present the results obtained by simulating both the
death-birth Moran process and the strategy adoption
process in well-mixed and scale-free networks of
players. Further, we present the results on how the
variation of network heterogeneity, measured using
the network assortativity, affects the evolutionary
stability of strategies. Finally, we discuss the re-
sults, presenting our conclusions.
2 Background
2.1 Game Theory
Game theory is the study of strategic decision
making [1]. Game theory was first developed as a
branch of micro-economics [1, 2]. However, later it
has been adopted in diverse fields of study, such as
evolutionary biology, sociology, psychology, politi-
cal science and computer science [2]. Game theory
has gained such wide applicability due to the preva-
lence of strategic decision making across different
fields of study. One of the key concepts of Game
theory is that of Nash equilibrium [24]. Nash equi-
librium suggests that there is an equilibrium state in
a strategic game, which neither player would bene-
fit deviating from. Nash equilibrium can be defined
for both pure strategy and mixed strategy scenar-
ios. Also, a strategy game could have multiple Nash
equilibria.
A formal definition for Nash equilibrium can be
given as follows. Let (S, f ) be a game with n play-
ers, where Si is the strategy set of a given player i.
Thus, the strategy profile S consisting of the strategy
sets of all players would be, S = S1 ×S2 ×S3 ....×Sn .
f = ( f1 (x), ....., fn (x)) would be the payoff function
85
for xεS. Suppose xi is th strategy profile of player i
and x−i be the strategy profile of all players except
player i. Thus, when each player iε1, ....., n chooses
strategy xi that would result in the strategy profile
x = (x1 , ...., xn ), giving a payoff of f (i) to that par-
ticular player. A strategy profile x∗ εS is in Nash
equilibrium if no unilateral deviation in strategy by
any single player would return a higher utility for
that particular player. Formally put,
∀i, xi εSi : fi (xi∗ , x−i
∗ ∗
) ≥ fi (xi , x−i ) (1)
2.2 Evolutionary Game Theory
Evolutionary game theory is an outcome of the
adaptation of game theory into the field of evo-
lutionary biology [3, 4]. Evolutionary biology is
based on the idea that an organism’s genes largely
contribute to its observable characteristics. Thus,
genes determine the fitness of an organism in a
given environment. Evolutionary game theory ar-
gues that the success of an organism depends on
its interaction with other organisms within a pop-
ulation. Thus, the fitness of an organism would be
dependent on its interaction with other organisms
in that population. In game theoretic terminology,
the interactions of players would be the strategies
and the fitness of each strategy would be the payoff.
As the population evolves over time, certain strate-
gies would be dominant while some other strategies
would be extinct. Thus, evolutionary game theory
provides a mathematical basis to model the evolu-
tionary process using concepts of game theory. On
the other hand, it opens a new dimension for game
theorists to observe how the evolution of strategies
in games would happen over time in populations
of players. Some of the critical questions asked in
EGT include; which populations/strategies are sta-
ble? When to individuals adopt other strategies?
and would it be possible for mutants to invade a
given population?
The equivalent concept to Nash Equilibrium in
evolutionary game theory, is evolutionary stability
[25]. If Nash Equilibrium can be considered as a
static equilibrium, evolutionary stability represents
a dynamic equilibrium of a strategy, over time. A
strategy is called evolutionarily stable if it has the
potential to dominate over any mutant strategy [3].
Evolutionary games are often modelled as iterative
games where a population of players play the same
Brought to you by | University of Sydney Library
Authenticated
Download Date | 4/19/17 6:19 AM
86 Dharshana Kasthurirathna, Mahendra Piraveenan, Shahadat Uddin

game iteratively in a well-mixed or a spatially dis- dilemma game are based on the assumption that
tributed environment [26]. each player chooses a strategy based on a proba-
bility distribution. In fact, pure strategies can be re-
strategy Iterated
2.2.1 is called evolutionarily
prisoner’s dilemma stable if it has the potential garded strategies
as a called
special finite-memory
case of mixed strategies
strategies[18], [28],
where where the
to dominate over any mutant strategy [3]. Evolutionary games current mixed strategy would be dependent on n number of
each strategy is chosen with the probability of one.
Prisoner’s
are often modelled dilemma is a games
as iterative game where
that isa found in of
population historical states between the two players.
players play the theory
same game Memory-one strategies [18, 28] are a special sub-
classical game [27].iteratively
Given theinpayoffa well-mixed
matrix or a When considering the previous state between two players
spatially distributed environment [26]. class of mixed strategies, where the current mixed
in Fig. 1, the inequality T>R>P>S should be satis- in a PD game, there could be four possible states. Namely
1) Iterated prisoner’s dilemma: Prisoner’s dilemma is a strategy CC, CD, of aDC gameand would
DD, where depend on the immedi-
C represents cooperation and
fied
gameinthata prisoner’s
is found indilemma
classical game. In other
game theory [27].words,
Given the ate D previous
represents interaction
defection,between
respectively.the two players in
Memory-one strategies
in the prisoner’s
payoff matrix in Fig. dilemma game, theT>R>P>S
1, the inequality highest mutual should be are represented by calculating the probabilities of cooperation
satisfied are
in aobtained
prisoner’s concern. On the other hand, Memory-one strategies
payoffs bydilemma
the players game.whenIn other words, in
both play- by a player in the next move, given the type of the previous
the prisoner’s dilemma game, the highest mutual payoffs are are interaction a specialisation of a more
of the player with thegeneral
same classopponent. of strate-
For example,
ers cooperate. However, if one player cooperates
obtained by the players when both players cooperate. However, giesa called strategyfinite-memory
(1,1,1,1) would strategies
imply that [18,the 28], where
Player A would
while
if one the other
player defects,while
cooperates the defector
the other would
defects,obtain a
the defector cooperate with player B, irrespective of the previous encounter
the current mixed strategy would be dependent on n
higher payoff.
would obtain The dilemma
a higher payoff. The is that the Nash
dilemma is thatEqui-
the Nash between Player A and B. Similarly, (0,0,0,0) would represent a
Equilibrium of this game, which occurs when both players number
playerofthat historical states between
always defects. Thus, the the puretwo players.
strategy cooperation
librium of this game, which occurs when both play-
defect, does not provide the optimum payoff for both the and defection can be thought of as a special case of memory-
ers defect, does not provide the optimum payoff for
players.
When considering the previous state between
one or finite memory strategies. By varying the probabilities
both the players. twoofplayers
cooperationin a PD under game,
eachthere
of thecould previousbe four pos- it is
encounters,
 sible states.
possible to Namely
define any CC,
number CD, of DC
mixed and DD,
strategies. where
Some of the
 
C represents cooperation and D represents defec-strategy
well-known memory-one strategies include the Pavlov
(1,0,0,1) and the general cooperator (0.935, 0.229, 0.266, 0.42)
tion, respectively. Memory-one strategies are rep-
strategy. General cooperator is the evolutionarily dominating
 resented
strategybythat calculating
evolved atthe lowprobabilities
mutation rates of ascooper-
demonstrated
  by Iliopoulos et al. [29]. Tit-for-Tat,
ation by a player in the next move, given the type is another wellofknowns
strategy where a player would only
the previous interaction of the player with the same cooperate if the opponent


cooperated in the previous interaction. This is signified by the

opponent.
probability Fordistribution
example,(1,1,0,0).
a strategy (1,1,1,1) would
imply that the Player A would cooperate with player
3) Zero-Determinant strategies: Zero-determinant strate-
   B, gies
irrespective
[21], [30] of arethe previous
a special encounter
sub-class of memory-onebetween strategies
Player
that A andrecently
have B. Similarly,
gained much (0,0,0,0) would
attention in represent
the literature and
media.that
a player ZD always
strategies denote Thus,
defects. a class the of memory-one
pure strategy strategies
that enable a player to unilaterally set the opponent’s payoff.
cooperation
Due to thisand defection canZD be strategies
thought have of asthe a ability
Figure 1. The payoff matrix of the prisoner’s inherent property,
special
to gaincase
a of memory-one
higher expected or finite
payoff againstmemory an strate-strategy.
opposing
dilemma game. T>R>P>S
Fig. 1: The payoff matrix of the prisoner’s dilemma game. gies. However,
By varyingfor a strategy to be evolutionarily
the probabilities of cooperationstable, it has to
T>R>P>S be stable against itself as well as the opponent strategies.
under each of the previous encounters, it is possible
In iterated prisoner’s dilemma(IPD), the pris- It has been shown that ZD strategies do not perform well
to define
against any number
itself. Due toofthis mixed strategies.
reason, ZD strategies Some have of been
oner’s dilemma game is iterated over many time- the well-known
demonstrated memory-one
to be evolutionarilystrategies
unstable include
[18], the
particularly
In iterated prisoner’s dilemma(IPD), the prisoner’s dilemma
steps,
game isover a population
iterated of playersover
over many time-steps, [17]. Each of Pavlov
a population against the Pavlov
strategy strategy.
(1,0,0,1) and the general cooperator
player would
players [17]. Eachplay a single
player would iteration
play a singleof the game
iteration of the
(0.935,ZD 0.229,
strategies0.266, are 0.42)
defined strategy.
using a set General coop- prob-
of conditional
game with
with its neighboursinin each
its neighbours each time-step.
time-step. Iterated prisoner’s
Iterated ability equations [21]. Suppose p , p , p and p denote the
dilemma game is widely used to model the autonomous deci- erator is the evolutionarily dominating strategy4that 1 2 3
prisoner’s dilemma game is widely used to model set of probabilities that a player would cooperate given that
sion making behaviour of self-interested players. It has been evolved at low mutation rates as demonstrated by
the autonomous decision making the player’s last interaction with the same opponent resulted
demonstrated that the topology of the behaviour of self- in Iliopoulos
network is significant et al. [29].
in the outcomes CC (p1Tit-for-Tat,
), CD (p2 ), DC is another
(p3 ) or DD well(p4 ). ZD
the evolution
interested of cooperation
players. of strategies
It has been in the IPD
demonstrated game
that the[23]. knowns
strategies are defined by fixing p2
strategy where a player would only cooper- and p3 to be functions of
For example, when the iterated prisoner’s dilemma game is
topology of the network is significant in the evolu- p1 and p4 , denoted by Eq. 2 and Eq. 3.
played among pure cooperation and pure defection strategies, ate if the opponent cooperated in the previous inter-
tion of cooperation
cooperation evolves to be of the
strategies
dominantin the IPD
strategy game
in a population action. This is signified by the probability distribu-
[23]. For example, when the iterated prisoner’s
of players that are distributed in a scale-free topology.
tion (1,1,0,0). p2 = 1
p (T − P ) − (1 + p4 )(T − R)
(2)
dilemma game isstrategies:
2) Memory-one played among pure cooperation
As opposed to pure strategies R−P
of cooperation
and pure defection and defection,
strategies,mixed strategies of
cooperation prisoner’s
evolves
dilemma game are based on the assumption that each player 2.2.3 Zero-Determinant strategies
to be the dominant strategy in a population of play-
chooses a strategy based on a probability distribution. In fact, (1 − p1 )(P − S) + p4 (R − S)
Zero-determinant p3 = strategies [21, (3)
ers that are distributed in a scale-free topology.
pure strategies can be regarded as a special case of mixed R− P 30] are a spe-
strategies where each strategy is chosen with the probability cial sub-class of memory-one strategies that have
of one. Memory-one
2.2.2 Memory-one strategies [18], [28] are a special sub-class recently
strategies It was
gained shownmuch by Press
attentionand Dyson
in the [21] that when
literature and playing
of mixed strategies, where the current mixed strategy of a game against the ZD strategy, the expected utility of an opponent
would
media. ZD strategies denote a class of memory-one
Asdepend
opposed on thetoimmediate previous interaction
pure strategies of coopera- between O can be defined using the probabilities p1 and p4 , while p2
strategies thatdefined
enableasa functions
player toofunilaterally set the
the two players in concern. On the other hand, Memory- and p3 are p1 and p4 . Eq. 4 gives the
tion and defection,
one strategies mixed strategies
are a specialisation of a more of general
prisoner’sclass of expected payoff of the opponent against the ZD strategy.

Brought to you by | University of Sydney Library

Authenticated
Download Date | 4/19/17 6:19 AM
EVOLUTIONARY STABLE STRATEGIES IN . . .
opponent’s payoff. Due to this inherent property,
ZD strategies have the ability to gain a higher ex-
pected payoff against an opposing strategy. How-
ever, for a strategy to be evolutionarily stable, it
has to be stable against itself as well as the oppo-
nent strategies. It has been shown that ZD strate-
gies do not perform well against itself. Due to this
reason, ZD strategies have been demonstrated to be
evolutionarily unstable [18], particularly against the
Pavlov strategy.
ZD strategies are defined using a set of con-
ditional probability equations [21]. Suppose p1 ,
p2 , p3 and p4 denote the set of probabilities that a
player would cooperate given that the player’s last
interaction with the same opponent resulted in the
outcomes CC (p1 ), CD (p2 ), DC (p3 ) or DD (p4 ).
ZD strategies are defined by fixing p2 and p3 to be
functions of p1 and p4 , denoted by Eq. 2 and Eq. 3.
p1 (T − P) − (1 + p4 )(T − R)
p2 = (2)
R−P
(1 − p1 )(P − S) + p4 (R − S)
p3 = (3)
R−P
It was shown by Press and Dyson [21] that when
playing against the ZD strategy, the expected utility
of an opponent O can be defined using the proba-
bilities p1 and p4 , while p2 and p3 are defined as
functions of p1 and p4 . Eq. 4 gives the expected
payoff of the opponent against the ZD strategy.
(1 − p1 )P + p4 R)
E(O, ZD) = (4)
(1 − p1 + p4 )
Here, P and R represent the payoffs earned
when both players defect and corporate, respec-
tively.
Hence, ZD strategies allow a player to unilat-
erally set the opponent’s payoff, effectively making
them extortionate strategies. In the simulations per-
formed here, we set the probabilities p1 and p4 as
0.99 and 0.01 respectively, as in the study done by
Adami and Hintze [18]. We then derive p2 and p3
to be 0.97 and 0.02, using the ZD conditional prob-
ability equations Eq. 2 and Eq. 3.
87
2.3 Evolutionary processes
We make use of two evolutionary processes in
the evolution of populations. The first one is a well-
known evolutionary process known as the death-
birth Moran process. The second one is the stochas-
tic strategy adoption process that was adopted from
the work of Santos et al. [23].
2.3.1 Death-birth Moran Process [22]
As the name suggests, in the death-birth Moran
process, a node is randomly selected for removal
at each time-step. Then, its replacement node is
selected from its neighbours based on a probabil-
ity proportional to the fitness of the neighbours. In
the case of the iterated prisoner’s dilemma game,
the fitness is equivalent to the accumulated payoff
of each node, averaged over its number of neigh-
bours. Then, the selected neighbour is replicated to
replace the node that is being removed. The new
node would have zero payoff yet it would still have
the same neighbours as the previous node that ex-
isted in the same topological space. This process
is continued over n number of time-steps to evolve
the entire population over time. Death-birth Moran
process is commonly used to emulate the evolution
of biological species where the strategies are ‘hard-
wired’ into the players. If the lifetime of a player
is significantly less than the time-span of evolution,
as with the case of biological evolution, death-birth
Moran process may effectively be used to simulate
the evolution of strategies(players) over time.
2.3.2 Stochastic strategy adoption process
Since Moran process maybe be more applica-
ble in the biological context where the players with
hard-wired strategies get replaced, it does not take
into account the individual payoff differences of
the node being replaced and the replicating node.
Thus, it maybe possible that the node being re-
placed would actually have a higher cumulative
payoff (fitness) than the replicating node. On the
other hand, in the social context, the time-span of
evolution of strategies could be considerably less
than the lifetime of a player. Hence, players would
be more inclined to adopt the apparently success-
ful strategy and survive without getting replaced
from the population. In order to model this kind
of social evolution, a stochastic strategy adoption
process can be applied. Such a process has been
Brought to you by | University of Sydney Library
Authenticated
Download Date | 4/19/17 6:19 AM
88
used in Santos et al. [23] to demonstrate the evolu-
tion of cooperation in IPD games with pure strate-
gies. We extend that method for mixed strategies
in this work. When employing this particular strat-
egy adoption process, a node going through evolu-
tion is not directly replaced. Instead, its strategy
could be updated by comparing its cumulative pay-
off with that of a stochastically selected neighbour.
As with the Moran process, in each time-step, a
node is marked for update. Then, a potential node
to compare it with is selected from its neighbours,
based on a probability proportional to the fitness
(accumulated payoff) of the neighbours. Then, the
probability of the marked node adopting the strat-
egy of its selected neighbour is calculated using the
following equation.
p = max{0, (Py − Px )/[k> (E (ZD, Pav) − E (Pav, ZD)]}
(5)
Where,
p - Probability that node X would adopt Y’s strategy
Px - Cumulative pay off of node X
Py - Cumulative pay off of node Y
k> - Maximum degree of X’s degree (kx ) and Y’s
degree (ky )
E (ZD, Pav) − The expected payoff of a ZD node
against a Pavlov node
E (Pav, ZD)– The expected payoff of a Pavlov node
against a ZD node
As shown above, the population update prob-
ability depends on the payoff difference of the
marked node and the selected neighbour node.
Also, the degree of those two nodes are used to nor-
malise the effect of degree differences. Still, the
cumulative payoff of the node with the higher de-
gree would be higher due to the fact that it would
have more interactions with other players. Thus,
this equation implicitly captures the network topol-
ogy in calculating the adoption probability. Due to
this reason, this particular strategy adoption process
can be used to study the topological effect on the
evolutionary stability of strategies.
2.4 Complex Networks
Complex Networks are self-organising net-
works that show non-trivial topological features [7].
Complex Network analysis provides a network per-
Dharshana Kasthurirathna, Mahendra Piraveenan, Shahadat Uddin
spective in analysing complex systems. Different
classes of complex networks have been defined to
model real-world complex systems such as social
and biological systems. In this work, we mainly
focus on two such network classes known as well-
mixed networks and scale-free networks that are
widely discussed in the network analysis literature.
2.4.1 Well-mixed Networks
Well-mixed networks has been one of the most
widely used network models in network analysis
since the inception of complex network science.
Also, they are widely used in modelling evolution-
ary games [29]. These networks are modelled, as-
suming that each node is connected to every other
node in the network. Due to this reason, well-mixed
networks can be regarded as a class of homoge-
neous networks. Therefore, it is possible to approx-
imate well-mixed networks to lattice networks with
each node having the identical number of neigh-
bours. However, most of the real-world networks
such as social networks and biological networks
do not demonstrate well-mixed behaviour, instead
being spatially distributed with non-homogeneous
topological features [7]. Still, well-mixed networks
provide a good reference model to compare with the
non-homogeneous networks [16].
2.4.2 Scale-free Networks
Scale-free networks demonstrate a network
topology that shows a power-law degree distribu-
tion [6]. In other words, the degree distribution of
the network would fit in a equation of the form y
= αx−γ . Here, γ is called the scale-free exponent.
The scale-free exponent is obtained by fitting a par-
ticular degree distribution into a power-law curve.
As the scale-free exponent increases, so does the
power-law nature of the degree distribution. Also,
if the correlation of the fitting is higher, such a de-
gree distribution would indicate that the respective
network closely resembles a scale-free network.
Scale-free nature is abundant in real-world net-
works, such as in social, biological and collabora-
tion networks [7]. Scale-free networks make per-
fect candidates to study the topological effect of a
population of players due to this reason. For ex-
ample, it has been shown that cooperation becomes
the dominant strategy in a scale-free topology due
to the heterogeneity of the network [16, 23]. More-
Brought to you by | University of Sydney Library
Authenticated
Download Date | 4/19/17 6:19 AM
EVOLUTIONARY STABLE STRATEGIES IN . . . 89

over, scale-free networks can be conveniently gen- of 1000 nodes that are connected via a lattice where
erated using the Preferential attachment growth each node is connected to 8 other random nodes.
model proposed by Barabasi and Albert [7]. Pref- Initially, the two strategies were distributed in a ran-
erential attachment model suggests that when a net- dom manner so that the ZD strategy would occupy
work grows in size, nodes with higher degree have a different fractions of the population (0.6 and 0.4)
higher probability of attracting new nodes. In other in each simulation run. Then, using the death-birth
words, there exists a degree-dependent ‘preference’ Moran process, the population was updated over
in creating links with the existing nodes, when a 150,000 time-steps to observe the evolution of the
new node joins the network. In this work, we use strategies.
scale-free networks generated using the preferential Afterwards, we changed the evolutionary pro-
attachment model for all our simulations. cess to the stochastic strategy adoption process used
by Santos et al. [23]. This enabled us to determine,
2.4.3 Network
2) Scale-free Assortativity
Networks: Scale-free networks demonstrate a random
whether nodes.
it isInitially, the two strategies
the population update were distributed
process or in
thea
network topology that shows a power-law degree distribution random manner so that the ZD strategy would occupy different
[6]. InAssortativity
other words, isthe the tendency
degree observed
distribution of the in net-
network network
fractions of topology
the population that(0.6
affects the
and 0.4) in evolutionary
each simulation sta-
run.
worksfit in
would where nodes
a equation mostly
of the form y connect
= αx . Here,
−γ
withγ similar
is called bilityusing
Then, of the
the strategies in concern.
death-birth Moran process, the population was
the scale-free exponent. The scale-free exponent is obtained by updated over 150,000 time-steps to observe the evolution of
nodes. Typically, this similarity is interpreted
fitting a particular degree distribution into a power-law curve.
in Then we replaced the well-mixed population
the strategies.
terms
As of the degrees
the scale-free exponent of nodes.so Assortativity
increases, does the power-lawhas with a non-homogeneous
nature Afterwards, we changed the scale-free
evolutionarynetwork
process towiththe
been offormally
the degreedefined
distribution.
as aAlso, if the correlation
correlation function of the
of 1000 nodes.
stochastic strategyAs with process
adoption the caseused ofbythe previous
Santos ex-
et al. [23].
fitting is higher, such a degree distribution would indicate that
excess degree distributions and link distribution
the respective network closely resembles a scale-free network. of This enabled the
periment, us to
ZDdetermine,
strategywhether
and theit Pavlov
is the population
strategy
update process or the network topology that affects the evolu-
a network [10, 13]. The formal definition of Assor- were stability
tionary assigned randomly
of the strategiesamong the nodes. With
in concern.
Scale-free nature is abundant in real-world networks, such
astativity makes
in social, use of
biological andnetwork topological
collaboration concepts
networks [7]. Scale- theThen
players spatially distributed in a scale-free net-
we replaced the well-mixed population with a non-
such as degree distribution p(k) and excess de-
free networks make perfect candidates to study the topological
work, both scale-free
homogeneous the death-birth
network Moran
with 1000 process
nodes. and the
As with
effect of a population of players due to this reason. For exam-
gree distribution q(k) for undirected networks [13].
ple, it has been shown that cooperation becomes the dominant
the case of the previous experiment, the ZD
strategy adoption process were applied separately strategy and the
Given in q(k), one cantopology
introduce thethequantity e j,k as Pavlov strategy were assigned randomly among the nodes.
strategy a scale-free due to heterogeneity of to observe thespatially
effect that they have on the evolution
With the players distributed in a scale-free network,
the network [16], [23]. Moreover, scale-free
the joint probability distribution of the remaining networks can
be conveniently generated using the Preferential attachment of strategies. The Fig. 3 depicts thestrategy
both the death-birth Moran process and the degreeadoption
distri-
degrees
growth of the
model two nodes
proposed at either
by Barabasi andend of [7].
Albert a randomly
Preferen-
process were applied separately to observe the effect that they
bution of the scale-free network used. As the figure
have on the evolution of strategies. The Fig. 3 depicts the
chosen link. Given these distributions, the assorta-
tial attachment model suggests that when a network grows in shows, the degree distribution shows a power-law
degree distribution of the scale-free network used. As the figure
size, nodes with higher degree have a higher
tivity of an undirected network is defined as: probability of
shows,
curvethethatdegree distribution with
is reminiscent showsthe a power-law curve that
typical scale-free
attracting new nodes. In other words, there exists a degree-
is reminiscent with the typical scale-free networks.
dependent ‘preference’[in creating links with the existing
] work, networks.
nodes, when a new node joins the( network. In this we use
1 )
scale-free networks
ρ= 2
σq ∑
generated using the preferential
jk e j,k − q j qk
model for all our simulations.
attachment
(6)
jk 400
3) Network Assortativity: Assortativity is the tendency
Number of nodes

observed in networks
where σq is thewhere nodes mostly
standard connectofwith
deviation similar
q(k). In
nodes. Typically, this similarity is interpreted in terms of the 300
this work,
degrees we Assortativity
of nodes. use networks has with varying defined
been formally assorta-
as
ativity values
correlation to observe
function of excesshow thedistributions
degree heterogeneity of
and link 200
distribution of a network
mixing patterns affect[10],
the[13]. The formal stability
evolutionary definition of
of
Assortativity makes use of network topological concepts such
asstrategies.
degree distribution p(k) and excess degree distribution q(k) 100
for undirected networks [13]. Given q(k), one can introduce
the quantity ej,k as the joint probability distribution of the
0
3 Methodology
remaining degrees of the two nodes at either end of a randomly
chosen link. Given these distributions, the assortativity of an 1 5 15 20
undirected network is defined as: Degree
Initially, we re-created the experimental results
obtained in the work
 
by Adami and Hintze [18], Figure
Fig. 3: The3.degree
The distribution
degree distribution of thenetwork
of the scale-free scale-free
used.
1 ∑
by mixing the ρ = Zero-determinant qk ) 
jk (ej,k − qjstrategy (6)
with the network used.
σq2
Pavlov strategy in a well-mixed population. This
jk
Next, we changed the initial distribution of the strategies in
further such aNext,
mannerwe thatchanged
the ZD strategy woulddistribution
occupy the majority
where σenabled us to confirm the theoretical re-
q is the standard deviation of q(k). In this work, we
the initial of the
of hubs. This was done by sorting the nodes according to their
sults
use presented
networks in the assortativity
with varying same workvalues usingtothe replica-
observe how strategies in such a manner that the ZD
degree and assigning the top 60% of the nodes with the ZD strategy
the
torheterogeneity
dynamics model, of mixing patterns that
suggesting affectthe
theZD
evolutionary
strategy would Inoccupy
strategy. the majority
this configuration, of hubs.
the initial averageThis
degreewas
of
stability of strategies. ZD nodes was measured to be 3.4 while the average degreede-
of
would be evolutionarily unstable against the Pavlov done by sorting the nodes according to their
Pavlov nodes was 1.8. The evolution of strategies was observed
strategy. To doIII. this,Mwe started with a population
ETHODOLOGY gree the
under anddeath-birth
assigningMoranthe top 60%as of
process welltheasnodes with
the strategy
adoption process. The experiment was repeated with the Pavlov
Initially, we re-created the experimental results obtained strategy occupying the majority of hubs.
in the work by Adami and Hintze [18], by mixing the Zero- Brought to you by | University of Sydney Library
determinant strategy with the Pavlov strategy in a well-mixed As the next step, we mixed the PavlovAuthenticated
strategy with the
population. This further enabled us to confirm the theoreti- general cooperator strategy and the cooperator strategy in
Download Date | 4/19/17 6:19 AM
cal results presented in the same work using the replicator separate scale-free networks of players. We tested the evolution
 90 Dharshana Kasthurirathna, Mahendra Piraveenan, Shahadat Uddin

1 1
ZD population fraction

ZD population fraction
0.8 0.4 0.8 0.4
0.6 0.6
0.6 0.6

0.4 0.4

0.2 0.2

0 0
0 20000 40000 60000 80000 0 20000 40000 60000 80000
Time-step Time-step
(a) W ell − mixed (b) Scale − f ree

Fig. 2:Figure
The evolution of ZD population
2. The evolution fraction against
of ZD population fractiontheagainst
Pavlovthe
strategy,
Pavlovinstrategy,
well-mixed and scale-free
in well-mixed populations. The
and scale-free
population is allowed to evolve under the death-birth Moran process with 0.1% replacement rate. The strategies are initially
populations. The population is allowed to evolve under the death-birth Moran process with 0.1%
distributed randomly, with the fraction of ZD nodes being 0.4 and 0.6, respectively.
replacement rate. The strategies are initially distributed randomly, with the fraction of ZD nodes being 0.4
and 0.6, respectively.
ZD strategy is a unique and inherent property of the stratety 1
itself or whether it is a more general behaviour that could occur
the ZD strategy. In this configuration, the initial lation fractions of ZD players were recorded. The
ZD population fraction
with other strategies as well. 0.8
average degree of ZD nodes was measured to be results were averaged over Random
40 independent runs.
Hubs
Finally, we observed
3.4 while the degree
the average evolution of the Pavlov
of Pavlov nodes was and 0.6
ZD strategies in non-homogeneous networks while the net-
1.8. The evolution
work heterogeneity of strategies
was gradually was observed
varying. To perform under
this
the death-birth Moran process as well as the
test, we generated a set of scale-free networks with varyingstrat- 4 Results
0.4

egy adoption
assortativity values process. Theaexperiment
by rewiring scale-free was repeated
network in a 0.2
probabilistic manner. Then, both strategies were distributed In certain figures in this section, we have limited
with the Pavlov strategy occupying the majority of
randomly in each scale-free population in such a manner that the number of 0 time-steps shown, when the strat-
hubs.
the ZD strategy would occupy 60% of the nodes. Afterwards, egy in concern 0becomes extinct 50000 100000
reasonably quickly.
the populations
As the were
nextallowed to evolve
step, we mixed over 150,000 strat-
the Pavlov time- Time-step
Fig.2 shows the evolution of the fraction of ZD
steps under the strategy adoption process and the remaining
egy with the general cooperator strategy and the co-
population fractions of ZD players were recorded. The results nodes
Fig. 4: when the ZDofstrategy
The evolution is mixed
ZD population with against
fraction the the
operatorover
were averaged strategy in separateruns.
40 independent scale-free networks of Pavlov
Pavlov strategy,
strategy in in well-mixed
scale-free populations of varying
and scale-free popu- initial
players. We tested the evolution of strategies with a configurations. The population
lations. The evolutionary is allowed
process used istothe
evolve under the
death-
death-birth Moran process with 0.1% replacement rate. In
random initial distribution of strategies and a strat- birthtwoMoran
IV. R ESULTS the initial process. As expected,
configurations, ZD strategy ZDis strategy
either assigned
egy distribution where the opponent strategy (GC gradually becomes extinct in a well-mixed
randomly or assigned more on hubs (in hubs initialisation,popu- the
In or
certain figures is
cooperator) in initially
this section,
assignedwe mostly
have limited the
on hubs. initial
lation. This confirms that in a homogeneous are
average degrees of ZD and Pavlov nodes net-3.4 and
number of time-steps shown, when the strategy in concern 1.8, respectively).
This enabled us to determine whether any observed
becomes extinct reasonably quickly. Fig.2 shows the evolution work, ZD strategy is evolutionarily unstable against
of the topological stability
fraction of ZD nodesofwhen
ZD strategy is a unique
the ZD strategy and
is mixed the Pavlov strategy that operates as a strong evo-
inherent
with the Pavlov property
strategy inofwell-mixed
the stratetyanditself or whether
scale-free it
popula- lutionarily stable strategy, as suggested by Adami
tions. The evolutionary process used is the death-birth
is a more general behaviour that could occur with Moran and Hintze
process. As expected, ZD strategy gradually becomes extinct in cannot survive[18].
whenMoreover, ZDisdoes
the population not tosurvive
allowed evolve under
other strategies as well. evendeath-birth
the in a non-homogeneous
Moran process. population distributed
a well-mixed population. This confirms that in a homogeneous
network, ZD strategy
Finally, is evolutionarily
we observed unstable
the evolution against
of the Pavlovthe in a scale-free network, when the same evolutionary
Pavlovand
strategy that operates as a strong evolutionarily stable Fig.5 depicts the scenario where a well-mixed population
ZD strategies in non-homogeneous networks process is applied.
strategy, as suggested by Adami and Hintze [18]. Moreover, of ZD and Pavlov strategies are allowed to interact with each
while the network
ZD does not survive even in heterogeneity
a non-homogeneouswas gradually
population otherNext,
over Fig.4
time, depicts
according
thetoevolution
the strategy adoption
of the ZD andprocess
varying.
distributed To perform
in a scale-free this test,
network, whenwethegenerated a set of
same evolutionary instead of the Moran
Pavlov strategies process. Here
in a scale-free too, ZD is gradually
non-homogeneous
processscale-free
is applied.networks with varying assortativity val- eradicated from the population. However, when the same
network of players,
evolutionary process isunder
applieddifferent initial config-
in a scale-free population of
uesFig.4
Next, by rewiring
depicts athe
scale-free
evolutionnetwork
of the in ZDa probabilis-
and Pavlov urations. The figure shows the evolution
players, ZD strategy manages to survive, as shown of in theFig6[a].
tic in
strategies manner. Then,
a scale-free both strategiesnetwork
non-homogeneous were distributed
of players, According to the figure, Pavlov strategy shows
ZD fraction when the strategies are initially dis- weak evolution-
under randomly
different initial configurations.
in each The figure in
scale-free population shows
suchthe a ary stability, failing to eradicate the ZD
tributed randomly as well as when more hubs are strategy completely,
evolution of the ZD fraction when the strategies
manner that the ZD strategy would occupy 60% are initially when the two strategies are initially assigned randomly. On the
distributed randomly as well as when more hubs are assigned assigned
other hand,with
whenthethe
ZDZDstrategy initially.
is initially As to
assigned thethefig-
majority
of ZD
with the the nodes.
strategy Afterwards,
initially. As the
the populations
figure depicts, were al-
Pavlov urehubs
of depicts, Pavlov clearly
as depicted dominates
in Fig.6[b], and eradicates
ZD manages to become the
clearlylowed to evolve
dominates over 150,000
and eradicates the ZD time-steps
strategy, under the
suggesting the ZD
weak strategy, suggesting
evolutionarily that irrespective
stable strategy over the Pavlovof the strategy,
that irrespective of the initial
strategy adoption processdistribution of strategies,
and the remaining popu-ZD becoming the dominant strategy in the network. However, as

Brought to you by | University of Sydney Library

Authenticated
Download Date | 4/19/17 6:19 AM
 0.2

ZD
0
000 0 20000 40000 60000 80000
Time-step
(b) Scale
EVOLUTIONARY STABLE − f ree
STRATEGIES IN . . . 91
gainst the Pavlov strategy, in well-mixed and scale-free populations. The
h Moran process with 0.1% replacement rate. The strategies are initially
being 0.4 and 0.6,initial
respectively.
distribution of strategies, ZD cannot survive players, ZD strategy manages to survive, as shown
when the population is allowed to evolve under the in Fig6[a]. According to the figure, Pavlov strat-
death-birth Moran process. egy shows weak evolutionary stability, failing to
e stratety 1 eradicate the ZD strategy completely, when the two
uld occur strategies are initially assigned randomly. On the
ZD population fraction

0.8 Random other hand, when the ZD is initially assigned to the

Hubs
vlov and 0.6 majority of hubs as depicted in Fig.6[b], ZD man-
the net- ages to become the weak evolutionarily stable strat-
orm this 0.4
varying egy over the Pavlov strategy, becoming the domi-
ork in a 0.2 nant strategy in the network. However, as the same
stributed figure shows, when the Pavlov strategy is initially
nner that 0 assigned to the majority of hubs, it behaves as a
terwards, 0 50000 100000
00 time- Time-step
strong evolutionarily stable strategy, wiping out the
emaining ZD population. This suggests that under the strat-
he results Fig. 4: The evolution of ZD population fraction against the egy adoption evolutionary process, the evolutionar-
Figure 4. The evolution of ZD population fraction
Pavlov strategy, in scale-free populations of varying initial
against
configurations. Thethe Pavlov strategy,
population is allowedintoscale-free
evolve under the ily unstable ZD strategy may not only survive, but
populations
death-birth Moran of varying
process initial
with 0.1% configurations. The In
replacement rate. may even become the more prominent strategy in a
the two initial configurations, ZD strategy is either
population is allowed to evolve under the assigned non-homogeneous population of players.
randomly or assigned more on hubs (in hubs initialisation, the
mited the death-birth Moran process with 0.1% replacement Then, we mixed the Pavlov strategy with the
initial average degrees of ZD and Pavlov nodes are 3.4 and
concern rate. In the two initial configurations, ZD strategy
1.8, respectively). GC and the cooperator strategies respectively, in
evolution is either assigned randomly or assigned more on
is mixed a Scale-free population. Fig.7 and Fig. 8 show
e popula-
hubs (in hubs initialisation, the initial average the evolution of the GC, cooperator, defector and
h Moran degrees of ZD and Pavlov nodes are 3.4 and 1.8, Tit-for-Tat strategy fractions over time, when those
extinct in cannot survive when the population is allowed to evolve under
respectively). strategies are initially placed randomly or mostly on
ogeneous the death-birth Moran process.
ainst the 1
hubs. As the figures
1
depict, the opposing strategies
ly stable Fig.5 depicts the scenario where a well-mixed population too may survive or dominate the population based
Remaining ZD fraction
ZD population fraction

Moreover, of ZD and Pavlov0.8 strategies are allowed to interact with each 0.8
on the initial distribution of the strategies, when the
other over time, according to the 0.4strategy adoption process
opulation 0.6 population is updated using the strategy adoption
lutionary instead of the0.6Moran process. Here too, ZD is gradually 0.6
eradicated from the population. However, when the same evolutionary process. This suggests that topological
evolutionary process
0.4 is applied in a scale-free population of influence on the0.4evolutionary stability is not limited
d Pavlov players, ZD strategy manages to survive, as shown in Fig6[a]. to the ZD strategy, but may apply to other strategies
f players, According to the0.2 figure, Pavlov strategy shows weak evolution- 0.2
hows the ary stability, failing to eradicate the ZD strategy completely, as well.
initially 0
when the two strategies are initially assigned randomly. On the Finally, we 0tested the evolution of the ZD pop-
assigned other hand, when0the ZD5000 10000assigned
is initially 15000to the
20000majority -0.5 -0.3 -0.1 0.1 0.3 0.5 0.7 0.9
Time-step ulation against the Pavlov strategy while the het-
Assortativity
, Pavlov of hubs as depicted in Fig.6[b], ZD manages to become the
uggesting weak evolutionarily stable strategy over the Pavlovagainst
strategy, erogeneity of the networks are gradually changed.
Fig. 5: The evolution of ZD population fraction the Fig. 9: The evolution of ZD population fraction against the
gies, ZD becoming
Figure
Pavlov the5.dominant
strategy,The strategyofinZD
in aevolution
well-mixed thepopulation
network.The
population. However,
fraction as Fig.9
population
shows the variation of the remaining ZD frac-
Pavlov strategy, in scale-free populations with varying network
is allowed to evolve
against under the
the Pavlov strategy
strategy, inadoption process. The
a well-mixed tion after 150,000
assortativity values.time-steps under
The population the strategy
is allowed to evolve under
strategies are initially distributed randomly,
population. The population is allowed to evolve with the fraction adoption
the process.
strategy As
adoption the figure
process. shows,
The there
results are exits
averaged over
of ZD nodes being 0.4 and 0.6, respectively. 40 independent
a negative runs. between the network assor-
correlation
under the strategy adoption process. The strategies
are initially distributed randomly, with the fraction tativity and the remaining ZD fraction. The actual
of ZD nodes being 0.4 and 0.6, respectively. Pearson correlation value of the two series is -0.85,
the same figure shows, when the Pavlov strategy is initially suggesting a strong negative correlation. Network
would influence the decisions made by individual entities, and
assigned to the majority of hubs, it behaves as a strong assortativity is a remain
many questions measure of the similarity
answered or how
in regards to the topology
Fig.5 depicts
evolutionarily the scenario
stable strategy, wipingwhere
out thea ZD
well-mixed
population. homogeneity
can influenceofthethe mixingindividual
strategies patterns players
of the nodes.
will choose over
Thispopulation of ZD
suggests that andthe
under Pavlov strategies
strategy adoptionareevolutionary
allowed time.
Therefore, this result suggests that the effect of net-
process, the evolutionarily
to interact with each unstable
other overZDtime,
strategy may nottoonly
according
survive, work Intopology on we
this work, theattempted
evolutionary stability
to evaluate howofthe
a network
the strategy adoption process instead of the strategy
but may even become the more prominent Moran in strategy
topologyincreases, as the network
of a population of playersbecomes more
affect the evolutionary
a non-homogeneous population of players.
process. Here too, ZD is gradually eradicated from stability of a strategy. In particular, we focused on a class
heterogeneous.
Then, we mixed However,
the population. the Pavlovwhen
strategy
thewith
samethe GC and the
evolution- of strategies known as Zero-determinant strategies, which has
cooperator strategies respectively, in a Scale-free population. been demonstrated to be evolutionarily unstable against the
ary process is applied in a scale-free population of
Fig.7 and Fig. 8 show the evolution of the GC, cooperator, de- Pavlov strategy.
fector and Tit-for-Tat strategy fractions over time, when those
strategies are initially placed randomly or mostly on hubs. As Based on the results gathered from this research, we
Brought to you by | University of Sydney Library
the figures depict, the opposing strategies too may survive or suggest that network topology hasAuthenticated
an effect on whether a
dominate the population based on the initial distribution of the particular strategyDownload
is evolutionarily stable
Date | 4/19/17 6:19 or
AMnot. However,
strategies, when the population is updated using the strategy the topologically influenced evolutionary stability is a weak
 92 Dharshana Kasthurirathna, Mahendra Piraveenan, Shahadat Uddin
1
1 1
1
1 1
fraction

fraction
fraction

fraction
0.8
0.8 0.8
0.8
fraction

fraction
0.8 0.8
0.6
0.6 0.6
0.6 ZD
ZD at
at Hubs
population

population
Hubs
population

population
0.6 0.6 Pavlov
Pavlov at Hubs
ZD at Hubs
population

population
0.4
0.4 0.4
0.4 Pavlov at Hubs
0.4 0.4
0.2
0.2 0.2
0.2
ZD

ZD
ZDZD

ZDZD
0.2 0.2
0
0 0
0
0 0
0 50000
50000 100000
100000 150000
150000 0 0
0 50000
50000 100000
100000 150000
150000
0 50000 100000
Time-step
Time-step 150000 0 50000 100000
Time-step
Time-step 150000
(a) Time-step (b) Time-step
Randomly −
(a) Randomly − initialised
initialised Hubs −
(b) Hubs − initialised
initialised
(a) Randomly − initialised (b) Hubs − initialised
Fig.
Fig. 6:
6: The
The evolution
evolution ofof ZD
ZD population
population fraction
fraction against
against the
the Pavlov
Pavlov strategy,
strategy, in
in scale-free
scale-free populations
populations of
of varying
varying initial
initial
Fig. 6: Figure
configurations.
The 6.
The The
evolution evolution
population
of ZD is of ZD
allowed
population population
to evolve
fraction fraction
under
againstthe
theagainst
strategy
Pavlov the Pavlov
adoption
strategy, strategy,
process.
in (a) in
scale-freeZDscale-free
and populations
Pavlov
populations
configurations. The population is allowed to evolve under the strategy adoption process. (a) ZD and Pavlov strategies strategies
of ofinitial
varyinginitially
initially
distributed randomly
configurations.
varying
distributed The (b)
initial
randomly Majority
population is of hubs
allowed
(b) configurations.
Majority of are
hubsThe
are either
to evolve assigned
underisthe
population
either assigned with
allowed
with ZD
strategy
ZDto or Pavlov
oradoption
evolve (the
Pavlov under
(the initial
process.
the(a)
initial average
ZD and
strategy
average degrees
Pavlovof
adoption
degrees hub
hub and
strategies
ofprocess. non-hub
initially
and (a)
non-hub
strategies
distributedare 3.4 and
randomly 1.8,
and(b) respectively).
1.8,Majority ofinitially
hubs aredistributed
either assigned with ZD
strategies are
ZD 3.4 Pavlov
and respectively).
strategies randomly (b)orMajority
Pavlov (the initialare
of hubs average
eitherdegrees of with
assigned hub and
ZDnon-hub
or
strategies are 3.4 and 1.8, respectively).
Pavlov (the initial average degrees of hub and non-hub strategies are 3.4 and 1.8, respectively).
1 1

fraction
1 1

fraction
1 1
fraction
fraction
fraction

0.8
0.8 0.8
0.8
fraction

population
0.8 0.8
population

0.6 0.6
population

0.6 Random 0.6 Random

population

Random Random
population

0.6 Hubs
Hubs
Random 0.6 Hubs
Hubs
Random
population

0.4
0.4 Hubs 0.4
0.4 Hubs
Coorperator

0.4 0.4
Coorperator

0.2 0.2
Coorperator

0.2 0.2
GC
GCGC

0.2 0.2
0
0 0
0
0 0
0 50000
50000 100000
100000 150000
150000 0 0
0 50000
50000 100000
100000 150000
150000
0 50000 100000
Time-step
Time-step 150000 0 50000 100000
Time-step
Time-step 150000
(a)
(a) GC − Time-step
GC − vs −P
vs − P avlov
avlov (b) Time-step
(b) Cooperator − vs
Cooperator − vs −
−PP avlov
avlov
(a) GC − vs − P avlov (b) Cooperator − vs − P avlov
Fig.
Fig. 7:
7: The
The evolution
Figure 7.
evolution of GC
GC and
of The cooperator
evolution
and of GC
cooperator strategies competing
and cooperator
strategies against
against the
competingstrategies Pavlov
Pavlov strategy,
thecompeting againstin
strategy, scale-free
inthe Pavlovpopulations
scale-free strategy, inof
populations of varying
varying
initial
Fig. 7: configurations.
The evolution The
of
scale-freeThe
initial configurations. GCpopulation
and
populations
population is allowed
cooperator
ofisvarying to
allowedinitialevolve
strategies under
competing
to evolve the strategy
against
configurations. the adoption
Pavlov process.
strategy,
The population
under the strategy in In the two
scale-free
is allowed
adoption process. to two
In the initial configurations,
populations
evolve under
initial of varying
the
configurations,
the
the competing
initial strategy
configurations.
competing is
is either
The
strategy assigned
population
either randomly
is allowed
assigned or assigned
to evolve undermore on
on hubs
the strategy (in hubs
adoption initialisation,
hubsprocess. In the the
twoinitial
initialaverage degrees
configurations,
of
thecompeting
strategy
competingnodes
adoption
and process. Inrandomly or
the two1.8, assigned
initial more
configurations, hubs
the(in
competinginitialisation,
strategy istheeither
initial average
assigned degrees
of competing nodes andisPavlov
strategy either nodes
Pavlov nodes are
assigned 3.4
3.4 and
arerandomly respectively).
or assigned
and 1.8, more on hubs (in hubs initialisation, the initial average degrees
respectively).
of competingrandomly
nodes or
andassigned more are
Pavlov nodes on hubs (in1.8,
3.4 and hubs initialisation, the initial average degrees of competing nodes
respectively).
and Pavlov nodes are 3.4 and 1.8, respectively).
1
1 1
1
fraction
fraction

fraction
fraction

1 1
fraction
fraction

0.8
0.8 0.8
0.8
0.8 0.8
population
population

population
population

0.6
0.6 Random 0.6
0.6 Random
population
population

Random Random
0.6 Hubs
Hubs
Random 0.6 Hubs
Hubs
Random
0.4
0.4 Hubs 0.4
0.4 Hubs
0.4 0.4
Tit-for-Tat
Defector

Tit-for-Tat
Defector

0.2
0.2 0.2
0.2
Tit-for-Tat
Defector

0.2 0.2
0
0 0
0
0 0
0 50000
50000 100000
100000 150000
150000 0 0
0 50000
50000 100000
100000 150000
150000
0 50000 100000
Timestep
Timestep 150000 0 50000 100000
Timestep
Timestep 150000
(a) Timestep (b) TTimestep
(a) Def ector −
Def ector vs −
− vs −PP avlov
avlov (b) T
T it −f
it − f or −T
or − − vs
at −
at −P
vs − P avlov
avlov
(a) Def ector − vs − P avlov (b) T it − f or − T at − vs − P avlov
Fig.
Fig. 8:
8: The evolution
TheFigure
evolution of
of Defector
Defector and Tit-for-tat strategies competing against the
the Pavlov strategy, in
in scale-free populations
of 8. The evolutionand Tit-for-tatand
of Defector strategies competing
Tit-for-tat against
strategies competing Pavlov strategy,
against the Pavlov scale-free
strategy, populations
in
Fig.varying
of 8: Theinitial
varying configurations.
evolution
initial of DefectorThe
configurations. andpopulation
The Tit-for-tatis
population allowed
allowed to
isstrategies evolve
evolve under
competing
to the
against
under strategy
thethe Pavlovadoption
strategy strategy,process.
adoption In
In the
in scale-free
process. two
two initial
the populations
initial
of scale-free
configurations, the
varying initial
configurations,
populations
competing
theconfigurations.
of varying
competing strategy
The
strategy is
is either
populationinitial
either assigned
configurations.
is allowed
assigned randomly or
to evolve
randomly
The population
or assigned
under
assigned more
the onishubs
strategy
more
allowed
on hubs (in toprocess.
(in hubs
adoptionhubs
evolve under
In the the
initialisation,
initialisation, the
the initial
two initial
average
average degrees
configurations,
degrees of
strategy
the adoptionnodes
of competing
competing and
process.
strategy
nodes and Pavlov
isIn
either
Pavlov nodes
the two
assigned
nodes are
initial 3.4
3.4 and
and 1.8,
arerandomly respectively).
configurations, the competing
or assigned
1.8, more on hubs
respectively). strategy is either
(in hubs assignedthe initial
initialisation,
average degrees
randomlyof competing nodes
or assigned and
more onPavlov nodes
hubs (in hubsareinitialisation,
3.4 and 1.8, respectively).
the initial average degrees of competing nodes
and Pavlov nodes are 3.4 and 1.8, respectively).

Brought to you by | University of Sydney Library

Authenticated
Download Date | 4/19/17 6:19 AM
 EVOLUTIONARY STABLE STRATEGIES IN . . . 93

5 Discussion tionary process used. When using the death-birth

Moran process to evolve the population, topology
does not seem have a significant effect on the evo-
1
lutionary stability of strategies. However, when
Remaining ZD fraction

0.8 the strategy adoption process suggested by San-

tos et al. [23] is applied, topology does have sig-
0.6
nificant effect on evolutionary stability of a strat-
0.4 egy within a population. Strategy adoption pro-
cess takes into account the cumulative payoff of
0.2 each node in determining whether a strategy should
0 be replaced or not. Therefore, this result suggests
0000 -0.5 -0.3 -0.1 0.1 0.3 0.5 0.7 0.9 that an evolutionarily unstable strategy could sur-
Assortativity vive when they occupy the hubs surrounded by leaf
against the Figure
Fig. 9: The 9. evolution
The evolution
of ZDofpopulation
ZD population against thenodes assigned with the evolutionarily stable strat-
fraction
fraction
population Pavlov strategy,
against inthescale-free populations
Pavlov strategy, with varying networkegy. In a heterogeneous network of players, hubs
in scale-free
process. The assortativity values. The population is allowed
populations with varying network assortativity to evolve undertend to have more strategic interactions with their
the fraction the strategy adoption process. The results are averaged over
40 values. The population
independent runs. is allowed to evolve under opponents compared to leaf nodes. Thus, a hub
the strategy adoption process. The results are with an evolutionarily unstable strategy would con-
averaged over 40 independent runs. tinue to be irreplaceable by the neighbouring nodes’
strategies, as it would continue to have a higher pay-
y is initially would influence the decisions made by individual entities, andoff compared to its immediate neighbours.
as a strong Network game theory has gained prominence as
many questions remain answered in regards to how topology
population. aninfluence
can interdisciplinary field.individual
the strategies Complexplayers
networks
will are in- over The significance of the evolutionary process
choose
evolutionary creasingly used to model social systems, while the
time. may have implications in the real-world networks
ay not only cognitive decision making processes of individuals of strategic players. Moran process would be more
t strategy in In this work, we attempted to evaluate how the network
topology of a population of players affect the evolutionaryappropriate in the biological context where the life-
in such systems is modelled by game theory. It
is obvious
stability of a that the topological
strategy. In particular, connection
we focused on a classtime of a player is significantly less than the evo-
patterns
GC and the ofwould
strategies known the
influence as Zero-determinant
decisions madestrategies, which haslutionary time-span. It could be effectively used to
by individual
population. been
operator, de- entities, and many questions remain answered in re- themodel the evolution of species where the strategies
demonstrated to be evolutionarily unstable against
Pavlov strategy.
when those gards to how topology can influence the strategies are ‘hard-wired’ to the players and the evolution
on hubs. As Based onplayers
individual the results gatheredover
will choose from this research, wehappens through the replacement of players with
time.
y survive or suggest that network topology has an effect on whether athe replicas of better performing players. However,
ution of the In this
particular work,isweevolutionarily
strategy attempted tostableevaluate howHowever,
or not. the
the strategy thenetwork topology
topologically of a population
influenced evolutionaryof players is a weakin the social context, the evolution of strategies may
stability affect
topological evolutionary
the evolutionarystabilitystability
and not of a strong evolutionary
a strategy. stability.be driven by the adoption of strategies by the play-
In partic-
d to the ZD In other words, the stable strategy would not be able toers based on the performance of their neighbouring
ular, we focused on a class of strategies known
. completely eradicate the competing strategy and the competingplayers. In other words, a stochastic strategy adop-
as Zero-determinant
strategy would be able tostrategies, which
survive within the has beenof the
confines
population demonstrated to be evolutionarily unstable against tion process could be used to model the evolution
network.
eity of the of strategies when the lifetime of a player maybe
variation of the Pavlov strategy.
We also identified that the topological effect of evolu-considerably larger than the time-span of evolution.
ps under the tionaryBased
stability
on isthedetermined by the evolutionary
results gathered from this re- processExamples of such situations involve the interactions
here exits a used. When
search, weusing
suggestthe death-birth
that network Moran process
topology hastoan
evolve
ef- thethat happen in corporate sector and financial mar-
vity and the population, topology does not seem have a significant effect
ion value of onfect
the on whether astability
evolutionary particular strategy isHowever,
of strategies. evolutionar-
when thekets. There, it is often observable that the players
correlation. strategy adoption process suggested by Santos et influ-
ily stable or not. However, the topologically al. [23] iscontinually adopt the strategies of other players, in
arity or the enced topology
applied, evolutionarydoes stability is a weak
have significant evolutionary
effect on evolutionarytheir struggle to survive. Thus, the strategy adop-
. Therefore, stability
stability and not a strong evolutionary stability.adoption
of a strategy within a population. Strategy In
logy on the process takes into account the cumulative payoff of eachtion evolutionary update process may be more rel-
the network other words, the stable strategy would not be able to
node in determining whether a strategy should be replacedevant when the evolution of strategies is applied in
orcompletely
not. Therefore,eradicate the competing
this result strategy
suggests that and the
an evolutionarily the social context. Accordingly, topological effect
competing
unstable strategy
strategy couldwould
survivebe when
able tothey
survive
occupywithin
the hubson the evolutionary stability of strategies may be
surrounded
the confines by of leaf
thenodes
network.assigned with the evolutionarily
stable strategy. In a heterogeneous network of players, hubsmore prevalent in the social context, compared to
e as an in- tend toWe alsomore
have identified
strategicthat the topological
interactions effect
with their of
opponents the biological context.
asingly used compared
evolutionaryto leafstability
nodes. Thus, a hub with
is determined by antheevolutionarily
evolu-
sion making unstable strategy would continue to be irreplaceable by the
ed by game neighbouring nodes’ strategies, as it would continue to have a
ion patterns higher payoff compared to its immediate neighbours. Brought to you by | University of Sydney Library
Authenticated
Download Date | 4/19/17 6:19 AM
94
Further, it is important to note that not only
the topology, but also the initial distribution of the
strategies within the network too plays a significant
role in shaping the evolution of the strategies. For
instance, when the evolutionarily unstable strategy
occupies hubs as opposed to the leaf nodes at the
initiation of the evolution, it even manages to be-
come the more prominent strategy within the net-
work over time, resembling a weak evolutionarily
stable strategy.
Even though we mainly focused on the ZD and
Pavlov strategies in this work, we could replicate
similar observations with other well-known strate-
gies such as the general cooperator and cooperator
strategies, competing against the Pavlov strategy.
This could mean that the variation of evolutionary
stability due to topological stability of strategies is
a more general phenomena that may be applica-
ble to most strategies that are competing with each
other. In addition, the topological effect on the evo-
lutionary stability may be observed for other games
other than the Prisoner’s dilemma game. For in-
stance, it would be interesting to observe how the
extensive form games, where the temporal dimen-
sion is built-in to the strategic decisions, are af-
fected by the topological placement of the players.
Such studies may be useful in accurately predicting
the evolutionary stability of strategies in scenarios
such as auctions, which can be modelled with ex-
tensive form games.
In conclusion, we could identify the following
three factors that determine the topological stability
of strategies in a non-homogeneous network. Net-
work topology, evolutionary process and the initial
distribution of the strategies. By varying these three
factors, an evolutionarily unstable strategy may be
able to survive and may even operate as a weak
evolutionarily stable strategy, in a population of
players connected in a non-homogeneous topology.
Based on our observations, the topological stabil-
ity of strategies may be more prevalent in the social
context of the evolution of strategies, in comparison
to the biological context.
Acknowledgment
The authors would like to thank Michael Harrè
of the Centre for Complex Systems Research at
the University of Sydney and Chistoph Adami and
Dharshana Kasthurirathna, Mahendra Piraveenan, Shahadat Uddin
Arend Hintze of Michigan State University for their
valuable comments and suggestions.
References
[1] J. Von Neumann and O. Morgenstern, “Game the-
ory and economic behavior,” 1944.
[2] E. Rasmusen and B. Blackwell, Games and infor-
mation. Cambridge, 1994, vol. 2.
[3] J. M. Smith and G. Price, “lhe logic of animal con-
flict,” Nature, vol. 246, p. 15, 1973.
[4] J. M. Smith, Evolution and the Theory of Games.
Springer, 1993.
[5] J. Bendor and P. Swistak, “Types of evolutionary
stability and the problem of cooperation.” Proceed-
ings of the National Academy of Sciences, vol. 92,
no. 8, pp. 3596–3600, 1995.
[6] A.-L. Barabási and R. Albert, “Emergence of scal-
ing in random networks,” science, vol. 286, no.
5439, pp. 509–512, 1999.
[7] R. Albert and A.-L. Barabási, “Statistical mechan-
ics of complex networks,” Reviews of Modern
Physics, vol. 74, pp. 47–97, 2002.
[8] G. Thedchanamoorthy, M. Piraveenan, S. Ud-
din, and U. Senanayake, “Influence of vaccination
strategies and topology on the herd immunity of
complex networks,” Social Network Analysis and
Mining, vol. 4, no. 1, pp. 1–16, 2014.
[9] M. Prokopenko, M. Piraveenan, and P. Wang, “On
convergence of dynamic cluster formation in multi-
agent networks,” Advances in Artificial Life, pp.
884–894, 2005.
[10] M. E. J. Newman, “Mixing patterns in networks,”
Physical Review E, vol. 67, no. 2, p. 026126, 2003.
[11] M. Piraveenan, M. Prokopenko, and A. Y. Zomaya,
“Local assortativeness in scale-free networks,” Eu-
rophysics Letters, vol. 84, no. 2, p. 28002, 2008.
[12] ——, “Assortative mixing in directed biological
networks,” IEEE/ACM Transactions on computa-
tional biology and bioinformatics, vol. 9(1), pp.
66–78, 2012.
[13] R. V. Solé and S. Valverde, “Information theory
of complex networks: on evolution and architec-
tural constraints,” in Complex Networks, ser. Lec-
ture Notes in Physics, E. Ben-Naim, H. Frauen-
felder, and Z. Toroczkai, Eds. Springer, 2004,
vol. 650.
[14] M. Piraveenan, D. Polani, and M. Prokopenko,
“Emergence of genetic coding: an information-
theoretic model,” in Advances in Artificial Life.
Springer Berlin Heidelberg, 2007, pp. 42–52.
Brought to you by | University of Sydney Library
Authenticated
Download Date | 4/19/17 6:19 AM
EVOLUTIONARY STABLE STRATEGIES IN . . .
[15] M. Piraveenan, M. Prokopenko, and L. Hos-
sain, “Percolation centrality: Quantifying graph-
theoretic impact of nodes during percolation in net-
works,” PloS one, vol. 8, no. 1, p. e53095, 2013.
[16] F. C. Santos and J. M. Pacheco, “Scale-free net-
works provide a unifying framework for the emer-
gence of cooperation,” Physical Review Letters,
vol. 95, no. 9, p. 098104, 2005.
[17] D. B. Fogel, “Evolving behaviors in the iterated
prisoner’s dilemma,” Evolutionary Computation,
vol. 1, no. 1, pp. 77–97, 1993.
[18] C. Adami and A. Hintze, “Evolutionary instabil-
ity of zero-determinant strategies demonstrates that
winning is not everything,” Nature communica-
tions, vol. 4, 2013.
[19] D. Kasthurirathna, M. Piraveenan, and M. Harre,
“Influence of topology in the evolution of coordi-
nation in complex networks under information dif-
fusion constraints,” The European Physical Jour-
nal B, vol. 87, no. 1, pp. 1–15, 2014.
[20] D. Kasthurirathna, H. Nguyen, M. Piraveenan,
S. Uddin, and U. Senanayake, “Optimisation
of strategy placements for public good in com-
plex networks,” in Social Computing (SocialCom),
2014 International Conference on. IEEE, 2014,
p. in print.
[21] W. H. Press and F. J. Dyson, “Iterated prisoners
dilemma contains strategies that dominate any evo-
lutionary opponent,” Proceedings of the National
Academy of Sciences, vol. 109, no. 26, pp. 10 409–
10 413, 2012.
[22] P. A. P. Moran et al., “The statistical processes of
evolutionary theory.” The statistical processes of
evolutionary theory., 1962.
Dharshana Kasthurirathna is a PhD
candidate of the complex systems re-
search group, within the faculty of
engineering and information technolo-
gies at the University of Sydney. His
primary research interests are complex
networks, game theory, information
theory and distributed systems.
Dr. Mahendra Piraveenan is a lec-
turer affiliated to the complex systems
research group, within the faculty of
engineering and information tech-
nologies at University of Sydney. He
holds a Bachelor of Engineering in
computer systems engineering (first
class) from University of Adelaide and
a Ph.D from University of Sydney. His
research interests include complex systems, networked game
theory, social networks and infectious disease dynamics.
95
[23] F. Santos, J. Rodrigues, and J. Pacheco, “Graph
topology plays a determinant role in the evolution
of cooperation,” Proceedings of the Royal Society
B: Biological Sciences, vol. 273, no. 1582, pp. 51–
55, 2006.
[24] J. F. Nash et al., “Equilibrium points in n-person
games,” Proceedings of the national academy of
sciences, vol. 36, no. 1, pp. 48–49, 1950.
[25] J. Maynard Smith, “The theory of games and the
evolution of animal conflicts,” Journal of theoreti-
cal biology, vol. 47, no. 1, pp. 209–221, 1974.
[26] S. Le and R. Boyd, “Evolutionary dynamics of the
continuous iterated prisoner’s dilemma,” Journal
of theoretical biology, vol. 245, no. 2, pp. 258–267,
2007.
[27] A. Rapoport, Prisoner’s dilemma: A study in con-
flict and cooperation. University of Michigan
Press, 1965, vol. 165.
[28] D. P. Kraines and V. Y. Kraines, “Natural selec-
tion of memory-one strategies for the iterated pris-
oner’s dilemma,” Journal of Theoretical Biology,
vol. 203, no. 4, pp. 335–355, 2000.
[29] D. Iliopoulos, A. Hintze, and C. Adami, “Critical
dynamics in the evolution of stochastic strategies
for the iterated prisoner’s dilemma,” PLoS compu-
tational biology, vol. 6, no. 10, p. e1000948, 2010.
[30] A. J. Stewart and J. B. Plotkin, “Extortion and co-
operation in the prisoners dilemma,” Proceedings
of the National Academy of Sciences, vol. 109,
no. 26, pp. 10 134–10 135, 2012.
Dr Shahadat Uddin conducts research
in the area of network dynamics, com-
plex healthcare coordination and col-
laboration networks, and data analysis
and modelling. His research addresses
interdisciplinary issues to understand
the impact of network structure and
network dynamics on group perfor-
mance and coordination outcomes, in
complex, dynamic and distributed environments. He holds a
PhD in network science from the University of Sydney.
Brought to you by | University of Sydney Library
Authenticated
Download Date | 4/19/17 6:19 AM