# Evolutionary prisoner’s dilemma games coevolving on adaptive networks

Evolutionary prisoner’s dilemma games coevolving on adaptive networks Abstract We study a model for switching strategies in the prisoner’s dilemma game on adaptive networks of player pairings that coevolve as players attempt to maximize their return. We use a node-based strategy model wherein each player follows one strategy at a time (cooperate or defect) across all of its neighbours, changing that strategy and possibly changing partners in response to local changes in the network of player pairing and in the strategies used by connected partners. We compare and contrast numerical simulations with existing pair approximation differential equations for describing this system, as well as more accurate equations developed here using the framework of approximate master equations. We explore the parameter space of the model, demonstrating the relatively high accuracy of the approximate master equations for describing the system observations made from simulations. We study two variations of this partner-switching model to investigate the system evolution, predict stationary states and compare the total utilities and other qualitative differences between these two model variants. 1. Introduction Game theory is the study of strategic decision making and the analysis of mathematical models of conflict and cooperation between intelligent rational participants [1–3]. Game theory is used in a variety of disciplines, including biology, economics, political science, computer science and psychology. In classical game theory, rational actors make their choices to optimize their individual payoffs. That is, actors make strategic choices on a rationally determined evaluation of probable outcomes, or utility, considering the strategic analysis that the players’ opponents are making in determining their own choices. Evolutionary game theory [4–8], the application of game theory to evolving populations, has recently expanded from consideration of life forms in biology to various areas in social science. Evolutionary game theory is useful in this context by defining a framework of contests, strategies, behaviours and analytics in which competition can be modelled. The key point in evolutionary game theory is that the success of a strategy is not just determined by how good the strategy is in itself; rather, it is a question of how good the strategy is in the presence of other alternative strategies and of the distribution of those other strategies within a population. One of the great difficulties of Darwinian theory, recognized by Darwin himself, was the problem of altruism [9]; in particular, exploited cooperators are worse off than defectors. Hence, according to the basic principles of Darwinian selection, intuitively it seems almost certain that cooperation should go extinct. If the basis for selection is at the individual level, the phenomenon of altruism is often hard to interpret. And selection at the group level, or for the ‘greater good’, appears to violate the game theory assumption of individuals maximizing their own utility. Indeed, altruism is certainly not found to be the general case in nature. Nevertheless, altruistic behaviours can be found in many social animals and, indeed, can be fundamental for some species to survive [10]. The solution to this apparent paradox can be demonstrated in the application of evolutionary game theory to the prisoner’s dilemma game [11, 12], a game which tests the outcomes of ‘cooperating’ versus ‘defecting’. Cooperation is usually analyzed in game theory by means of a non-zero-sum game. The prisoner’s dilemma game, one of the most studied systems in all of game theory, is a standard example that shows why two completely rational individuals may not cooperate, even if it appears that it is in their best collective interest to do so. Within evolutionary game theory, the analysis of the prisoner’s dilemma is as an iterative game [13], with the repetitive nature affording competitors the possibility of retaliating or defecting based on the results of previous rounds of the game. There are a multitude of strategies which have been tested by the mathematics of evolutionary game theory and in computer simulations of contests (see, e.g. [14]), with the general conclusion that the most effective competitive strategies are typically cooperative with a reserved retaliatory response as necessary. The most famous and one of the most successful of these strategies is ‘Tit for Tat’, which carries out this approach by executing a simple algorithm [11, 15]. In recent years, games played on various random graphs and social networks have been investigated [16–25, 27, 28]. For example, the public goods game can be seen as a generalization of multiplayer N-player games and group interactions such as pattern formation and self-organization could be studied through this extension [29–31]. The studies devoted to evolutionary games on complex networks are extensive. Speaking generally, the goals of such studies typically include identifying which combination of game rules, dynamics and various network topologies can provide cooperation among selfish and unrelated individuals. In the present work, we focus on the dynamical role of players having the option to switch partners, as studied in the framework of adaptive networks that coevolve with the player strategies [32–37]. On a ‘coevolving’ or ‘adaptive’ network with a game, the vertices represent players and the edges denote the pairings, or game interactions, between players. Importantly, the spatial structures in a network could enable cooperators to form small groups or clusters to protect themselves against exploitation by defectors [25]. Studies have shown that the spatial structure can promote cooperation [26], but there is also evidence that suggests that spatial structure may not necessarily favour cooperation [20]. Other studies have elaborated on different aspects of cooperation on scale-free [38–40], square-lattice [41, 42], small-world [43–45], social and real-world networks [46–48] and multilayer networks [49]. Insofar as the player pairing network is part of a larger social setting, other notable rules encouraging cooperative behaviour are kin selection [50], group selection [51, 52], direct reciprocity [53], indirect reciprocity [54, 55], social diversity [56–58], voluntary participation [59–61], reputation [18] and frequency of breaking cooperator-defector bonds and rewiring [36, 37]. All of these have been studied as interesting mechanisms that may promote cooperation in evolutionary games. In this article, we focus our study on a simple coevolving network model for how players play a prisoner’s dilemma game with the ability to adapt by changing their strategies or switching partners if they are exploited by their neighbours, introduced in [19]. We are interested in exploring the cooperative level among the individuals as they organize into networks of cooperators. By using the technique of approximate master equations, we provide a more accurate approximation of the evolution of the network structure and player states with which to explore the parameter space of the model. We compare the existing analytical methods and our new approximation, providing qualitative and quantitative estimations of various network properties. In Section 2, we introduce the model. We then describe our more accurate approximation in Section 3. In Section 4, we provide numerical results and compare the two approximation methods. In Section 5, we introduce a variation to the model and compare the results of the two variants of the model. Finally, we make concluding remarks in Section 6. 2. Description of the Cooperative-Defective (CD)-switching model We study the partner switching model introduced by Fu et al. [19], where vertices of the coevolving network represent players, edges denote the pairing of individuals playing the game, and players adapting by changing strategies based on their current local information or switching partners. For simplicity, we initialize our simulations with an Erdős–Rényi network with $$N$$ nodes and $$M$$ edges, with each node assigned an initial state either as a cooperator ($$C$$) or defector ($$D$$). Each node uses a single strategy across all of its links, though that strategy can change over time as we describe below. That is, individual $$i$$ playing with all of her connections obtains an income   Pi=∑j∈NisiTPsj, where $$\mathscr{N}_{i}$$ is the neighbourhood set of $$i$$ and the 2-by-2 payoff matrix $$P$$ is   CD(C11+uD0u) with cost-benefit ratio $$u \in (0,1)$$ determining the relative outcomes in the payoff matrix. That is, if players $$i$$ and $$j$$ in a pairing both play $$C$$, then both receive the payoff $$1$$ from this pairing; if player $$i$$ plays $$C$$ and player $$j$$ plays $$D$$, then player $$i$$ gets payoff $$0$$ and player $$j$$ gets payoff $$1+u$$; and so on. Each time step of our microsimulation proceeds as follows. We uniformly at random pick one of the edges that connects a pair of players with different strategies, that is a $$CD$$ link, denoted by $$E_{ij}$$. (In Section 5, we study a variant of the model where both $$CD$$ and $$DD$$ links are considered.) With probability $$w$$ (specified as a parameter), the two nodes $$i$$ and $$j$$ connected by edge $$E_{ij}$$ consider updating their strategies; otherwise (i.e. with probability $$1-w$$), the edge $$E_{ij}$$ is rewired. When a node reassesses its strategy along the edge $$E_{ij}$$, the node has probability $$\phi$$ specified by the Fermi function to change its state, as first proposed by [62], as specified in detail below. When link $$E_{ij}$$ is rewired, the player with end state $$C$$ unilaterally drops the partnership with its neighbour with end state $$D$$ on the edge $$E_{ij}$$ and picks (uniformly at random) another player from the remaining population outside its immediate neighbourhood as its new partner. The left panel of Fig. 1 illustrates this rewiring process. Fig. 1. View largeDownload slide (Color online) Illustrations of the rewiring and strategy updating processes. Pick a discordant edge $$XY$$ with node $$X$$ in state $$C$$ and node $$Y$$ in state $$D$$. (Left) Rewiring occurs with probability $$1-w$$ with cooperating node $$X$$ dismissing its defecting neighbour $$Y$$ and rewiring to a random node $$Z$$ in the network, independent of the state of node $$Z$$. If $$Z$$ is in state $$D$$, then the number of $$CD$$ edges remains the same; but if $$Z$$ is cooperating (as in the picture), then the number of $$CD$$ edges has been reduced. (Right) Strategy updating occurs with probability $$w$$. Nodes $$X$$ and $$Y$$ compare their utilities, with one node copying the other as selected according to the Fermi function of the difference of utilities. In the example depicted here, suppose the cost-benefit ratio $$u = 0.5$$ and the Fermi function bias parameter $$\alpha=30$$. Then node $$X$$ has utility $$P_X=1$$, node $$Y$$ has utility $$P_Y=4$$, and the Fermi function gives $$\phi(s_{X} \leftarrow s_{Y}) = {1}/({1+\exp[\alpha(P_{X}-P_{Y})]}) = {1}/({1+\exp[30(1-4)]}) \approx 1$$, so that node $$X$$ is selected with probability close to 1 to imitate node $$Y$$’s strategy (state $$D$$). Fig. 1. View largeDownload slide (Color online) Illustrations of the rewiring and strategy updating processes. Pick a discordant edge $$XY$$ with node $$X$$ in state $$C$$ and node $$Y$$ in state $$D$$. (Left) Rewiring occurs with probability $$1-w$$ with cooperating node $$X$$ dismissing its defecting neighbour $$Y$$ and rewiring to a random node $$Z$$ in the network, independent of the state of node $$Z$$. If $$Z$$ is in state $$D$$, then the number of $$CD$$ edges remains the same; but if $$Z$$ is cooperating (as in the picture), then the number of $$CD$$ edges has been reduced. (Right) Strategy updating occurs with probability $$w$$. Nodes $$X$$ and $$Y$$ compare their utilities, with one node copying the other as selected according to the Fermi function of the difference of utilities. In the example depicted here, suppose the cost-benefit ratio $$u = 0.5$$ and the Fermi function bias parameter $$\alpha=30$$. Then node $$X$$ has utility $$P_X=1$$, node $$Y$$ has utility $$P_Y=4$$, and the Fermi function gives $$\phi(s_{X} \leftarrow s_{Y}) = {1}/({1+\exp[\alpha(P_{X}-P_{Y})]}) = {1}/({1+\exp[30(1-4)]}) \approx 1$$, so that node $$X$$ is selected with probability close to 1 to imitate node $$Y$$’s strategy (state $$D$$). When a strategy updating event occurs, nodes $$i$$ and $$j$$ each consider their play across all of their neighbours, observing their total payoffs $$P_{i}$$ and $$P_{j}$$. Then the strategy of node $$j$$ replaces that of $$i$$ along the edge $$E_{ij}$$—that is, $$i$$ copies $$j$$’s strategy—with probability given by the Fermi function   ϕ(si←sj)=11+exp⁡[α(Pi−Pj)], where $$\alpha$$ modifies the intensity of the bias towards the strategy with the higher payoff [63, 64]. Otherwise [that is, with probability $$\phi(s_{j} \leftarrow s_{i}) = 1 - \phi(s_{i} \leftarrow s_{j})$$], $$j$$ copies $$i$$. The value of $$1/\alpha$$ can be interpreted here as representing the amplitude of noise in the strategy updating process [66, 67]; that is, $$\alpha \rightarrow 0$$ ignores the payoffs while $$\alpha \rightarrow \infty$$ yields deterministic imitation of the node receiving the higher payoff. The right panel of Fig. 1 illustrates the strategy updating process. This partner switching evolutionary game stops when there are no discordant edges remaining in the network. That is, only $$CC$$ and $$DD$$ edges exist in the final state of the system, with the final network fissioning these two groups (cooperative and defective) into different components. In Fig. 2, we visualize a simulated network shortly before fission occurs, showing strong grouping of nodes by strategies. Fig. 2. View largeDownload slide (Color online) Visualization of the CD-switching coevolving network model shortly before the network fissions into two disconnected components, one including all of the remaining cooperator nodes (light/blue) and the other containing only defecting nodes (dark/red). This simulation contains $$N \,{=}\, 1,000$$ nodes and $$M \,{=}\, 5,000$$ edges, with cost-benefit ratio $$u \,{=}\, 0.5$$, strategy updating probability $$w \,{=}\, 0.1$$, and initial fraction of defectors $$\rho \,{=}\, 0.5$$. This visualization was created using the Yifan Hu layout in Gephi [65]. Fig. 2. View largeDownload slide (Color online) Visualization of the CD-switching coevolving network model shortly before the network fissions into two disconnected components, one including all of the remaining cooperator nodes (light/blue) and the other containing only defecting nodes (dark/red). This simulation contains $$N \,{=}\, 1,000$$ nodes and $$M \,{=}\, 5,000$$ edges, with cost-benefit ratio $$u \,{=}\, 0.5$$, strategy updating probability $$w \,{=}\, 0.1$$, and initial fraction of defectors $$\rho \,{=}\, 0.5$$. This visualization was created using the Yifan Hu layout in Gephi [65]. We note that recent work [68] studied a spatially embedded extension of the partner switching model in [19], introducing a range of rewiring distance and finding that a preference for global partner switching can coevolve with cooperation. However, in the present contribution we consider only random rewiring partner switching. 3. Semi-analytical methods of approximation We study this generalized coevolving network model with simulations and approximate model equations. The frameworks of mean field theory, pair approximation equations (PA) and approximate master equations (AME) have all been used effectively in similar settings. The PA equations for these dynamics were obtained by [19]. Among these three levels of approximation, AME can often be used to achieve the greatest accuracy [69], motivating us to develop the AME system here. For comparison with the AME system that we will specify, we first look at the PA equations derived in [19], describing the dynamics of the quantities $$N_{X}$$ and $$N_{XY}$$—where $$X,Y \in \{ C, D\}$$—that count the numbers of nodes and edges corresponding to the two node states. For example, $$N_{C}$$ denotes the number of cooperators and $$N_{CC}$$ denotes the number of $$CC$$ links in the network. While the node states and network topology coevolve, the total numbers of nodes ($$N$$) and edges ($$M$$) remain conserved by the specified dynamics, requiring $$N_{C} + N_{D} = N$$ and $$N_{CC} + N_{CD} + N_{DD} = M$$. The evolution of these counts is then obtained through a moment closure approximating triple counts $$N_{XYZ}$$—where $$X, Y, Z \in \{ C, D\}$$—in terms of the edge counts $$N_{CC}$$, $$N_{CD}$$ and $$N_{DD}$$. Specifically, $$N_{XXY}$$ is approximated by the product of $$XX$$ links and the average number of $$XY$$ links that an $$X$$ node has, that is $$N_{XY}/N_{X}$$, hence $$N_{XXY} = 2N_{XX}N_{XY}/N_{X}$$. The resulting PA equations [19] are as follows:   dNCdt =w⋅NCD⋅tanh⁡[α2(π¯C−π¯D)]dNCCdt =w⋅(NCDϕC→D−2NCDNCCNCϕD→C+NCDNCDNDϕC→D) +(1−w)⋅NCNNCDdNDDdt =w⋅(NCDϕD→C−2NCDNDDNDϕC→D+NCDNCDNCϕD→C), (3.1) where $$w$$ is the probability of strategy updating (versus edge rewiring), $$\overline{\pi}_{C} = 1 \cdot {2N_{CC}}/{N_{C}} + 0 \cdot {N_{CD}}/{N_{D}}$$ is the average utility of $$C$$ nodes, $$\overline{\pi}_{D} = (1+u) \cdot {N_{CD}}/{N_{D}} + u \cdot {2N_{DD}}/{N_{D}}$$ is the average utility of $$D$$ nodes, $$\phi_{C\to D} = 1/(1+\exp [\alpha (\overline{\pi}_{D} - \overline{\pi}_{C})])$$, $$\phi_{D\to C} = 1/(1+\exp [\alpha (\overline{\pi}_{C} - \overline{\pi}_{D})])$$, and $$\alpha$$ controls the intensity of selection. The AME method has been successfully used to approximate various processes on static and coevolving networks. The difficulty in deriving the AME system for the dynamics studied here is that the present transition probabilities depend on more than nearest neighbour states. For comparison, in the Susceptible-Infected-Susceptible (SIS) model the recovery rate of an infected (I) node and the probability of infection along a given SI edge are both specified as constant parameters. Similarly, in typical voter model dynamics, the probability for a node to change its state relies on its neighbours’ opinions. In contrast, in the partner switching evolutionary game studied here, the local information impacting the decision of a node to change its state includes its total utility and that of its neighbours, which depend on the states of the neighbours’ neighbours. Since the binary-state AME framework only includes the node states, degrees and neighbour states, we need estimate neighbours’ utilities in terms of these limited quantities. We will explain how we approximate these utilities after we introduce the equations below. Let $$C_{k,l}(t)$$ and $$D_{k,l}(t)$$, respectively, be the number of cooperating and defecting nodes of degree $$k$$ with $$l$$ defecting neighbours at time $$t$$. We note that the numbers of nodes using each strategy are then given by the zeroth moments of the $$C_{k,l}(t)$$ and $$D_{k,l}(t)$$ distributions, $$N_{C} = \sum_{kl}C_{k,l}$$ and $$N_{D} = \sum_{kl}D_{k,l}$$. Meanwhile, the first moments give the numbers of edges of each type: $$N_{CC} = \frac{1}{2} \sum_{kl}(k-l)C_{k,l}$$, $$N_{CD} = \sum_{kl}lC_{k,l} = \sum_{kl}(k-l)D_{k,l}$$ and $$N_{DD} = \frac{1}{2} \sum_{kl}lD_{k,l}$$. Importantly, while the node states and network topology coevolve, the total numbers of nodes and edges remain conserved by the specified dynamics, requiring $$N_{C} + N_{D} = N$$ and $$N_{CC} + N_{CD} + N_{DD} = M$$. The AME system of ordinary differential equations governing the $$C_{k,l}(t)$$ and $$D_{k,l}(t)$$ compartments is of course more complicated than the (relatively) compact PA equations (3.1). In the following paragraphs, we describe each of the terms that appear in this AME system:   dCk,ldt =w{ϕk,lD⋅(k−l)Dk,l−ϕk,lC⋅lCk,l +ϕCD←CC⋅γC(l+1)Ck,l+1−ϕCD←CC⋅γClCk,l +ϕCC←CD⋅βC(k−l+1)Ck,l−1−ϕCC←CD⋅βC(k−l)Ck,l} +(1−w){NCN[(l+1)Ck,l+1−lCk,l]+NCDN[Ck−1,l−Ck,l]}, (3.2)  dDk,ldt =w{−ϕk,lD⋅(k−l)Dk,l+ϕk,lC⋅lCk,l +ϕDD←DC⋅γD(l+1)Dk,l+1−ϕDD←DC⋅γDlDk,l +ϕDC←DD⋅βD(k−l+1)Dk,l−1−ϕDC←DD⋅βD(k−l)Dk,l} +(1−w){[(k−l+1)Dk+1,l−(k−l)Dk,l]+NCDN[Dk−1,l−Dk,l]}. (3.3) To describe the AME derivation, we focus on explaining the $$C_{k,l}$$ equation, as the effects of corresponding terms of the $$D_{k,l}$$ equation are similar. The first three lines of equation (3.2), premultiplied by $$w$$, are the effect of strategy updating, while the last line with $$1-w$$ are the effect of rewiring. To write the AME system, we track all counts flowing in and out of ‘centre’ class, $$C_{k,l}$$. As visualized in Fig. 3, there are six such flows for the $$C_{k,l}$$ compartment, yielding the first six terms in the $$C_{k,l}$$ equation. The first term describes the rate at which $$D_{k,l}$$ nodes change to $$C_{k,l}$$ through comparing its utility with one of its $$C$$ neighbours, with the Fermi function evaluation $$\phi^{D}_{k,l}$$ as estimated below. Similarly, the second term captures $$C_{k,l}$$ nodes changing strategy to become $$D_{k,l}$$ after comparing their utility with their $$D$$ neighbours, with Fermi function evaluation $$\phi^{C}_{k,l}$$ also estimated below. Fig. 3. View largeDownload slide (Color online) Illustration of transitions to and from the $$C_{k,l}$$ and $$D_{k,l}$$ compartments (cooperating and defecting nodes, respectively, with degree $$k$$ and $$l$$ defecting neighbours) in the system of AME due to players updating their strategies. For each compartment, only a subset of neighbours in the corresponding ego network are depicted here, to signal the essential changes corresponding to each flow. The term describing each flow is indicated above/below the corresponding line connecting compartments. Fig. 3. View largeDownload slide (Color online) Illustration of transitions to and from the $$C_{k,l}$$ and $$D_{k,l}$$ compartments (cooperating and defecting nodes, respectively, with degree $$k$$ and $$l$$ defecting neighbours) in the system of AME due to players updating their strategies. For each compartment, only a subset of neighbours in the corresponding ego network are depicted here, to signal the essential changes corresponding to each flow. The term describing each flow is indicated above/below the corresponding line connecting compartments. To estimate these Fermi function transition probabilities, we can directly compute the utility of the centre node, since we know the numbers and types of its neighbours; but we need to estimate neighbours’ utilities in terms of the limited information available in the AME framework. Recall that the payoff matrix of our prisoner’s dilemma game is   CD(C11+uD0u), where $$u \in (0,1)$$. For transitions by a strategy update out of the $$D_{k,l}$$ class, the centre node is of state $$D$$, with $$k$$ neighbours, $$l$$ of which are of state $$D$$. Hence we can compute the centre’s utility as   PD=l⋅u+(k−l)⋅(1+u). Since we do not have any information about the neighbours of the $$C$$ neighbours of these $$D_{k,l}$$ nodes, we approximate utility by estimating the numbers and types of the neighbours of a $$C$$ neighbour. Specifically, we denote the expected number of $$C$$ neighbours of this $$C$$ neighbour by $$\eta^{D}$$, and the expected number of $$D$$ neighbours of a $$C$$ neighbour by $$\beta^{D} + 1$$, where $$\eta^{D}$$ and $$\beta^{D}$$ will be estimated below and the $$+1$$ accounts for the $$D_{k,l}$$ node itself. The estimate of the utility of the $$C$$ neighbour thus becomes   P^C=ηD⋅1+(βD+1)⋅0 and we compare the centre node $$D_{k,l}$$’s utility with the estimate from one of its $$C$$ neighbours via the Fermi function, so that the estimated probability of the $$D_{k,l}$$ node changing its state in this consideration is   ϕk,lD=11+exp⁡[α(PD−P^C)]=11+exp⁡[α(lu+(k−l)(1+u)−ηD)]. Meanwhile, transitions by a strategy update out of the $$C_{k,l}$$ rely on the utility of a $$C_{k,l}$$ node,   PC=l⋅0+(k−l)⋅1, and estimating the utility of one of the $$D$$ neighbours via the numbers and types of its neighbours. We denote the expected number of $$C$$ neighbours of this $$D$$ neighbour (including the $$C_{k,l}$$ node in question) by $$\gamma^{C}$$ + 1, and the number of $$D$$ neighbours of this $$D$$ neighbour as $$\delta^{C}$$, both of which will be estimated below. With this notation, the expected utility of one of the $$D$$ neighbours is   P^D=(γC+1)⋅(1+u)+δC⋅u and the Fermi function estimate for the probability of the $$C_{k,l}$$ node changing its state in this comparison becomes   ϕk,lC=11+exp⁡[α(PC−P^D)]=11+exp⁡[α(k−l−(γC+1)(1+u)+δCu)]. The third–sixth terms in the $$C_{k,l}$$ equation are the effects of one of the neighbours changing its state and thus leading to a change of the total quantity of $$C_{k,l}$$. Similar to the above, in order to make estimates of the transition probabilities we need a variety of estimates of the numbers and types of the neighbours’ neighbours, which we denote as follows. We use $$\beta^{C}$$ to provide the expected number of $$D$$ neighbours of a $$CC$$ edge, as   βC=∑k,l(k−l)lCk,l∑k,l(k−l)Ck,l. Similarly, $$\beta^{D}$$ denotes the expected number of $$D$$ neighbours of the $$C$$ node in a $$CD$$ edge by   βD=∑k,ll2Ck,l∑k,llCk,l. Meanwhile, $$\gamma^{C}$$ and $$\gamma^{D}$$ estimate the numbers of $$C$$ neighbours of the $$D$$ node of a $$CD$$ edge or of a $$DD$$ edge, respectively:   γC=∑k,l(k−l)2Dk,l∑k,l(k−l)Dk,l,γD=∑k,ll(k−l)Dk,l∑k,llDk,l. Similarly, $$\delta^{C}$$ and $$\delta^{D}$$ provide the expected numbers of $$D$$ neighbours of the $$D$$ node of a $$CD$$ edge or of a $$DD$$ edge, respectively, while $$\eta^{C}$$ and $$\eta^{D}$$ give the numbers of $$C$$ neighbours of a $$CC$$ edge or the $$C$$ node of a $$CD$$ edge, respectively:   δC=∑k,l(k−l)lDk,l∑k,l(k−l)Dk,l,δD=∑k,ll2Dk,l∑k,llDk,l,ηC=∑k,l(k−l)2Ck,l∑k,l(k−l)Ck,l,ηD=∑k,ll(k−l)Ck,l∑k,llCk,l. To compute the corresponding $$\phi$$ transition probability in each term, let us first consider the Fermi function factors on the second line in equation (3.2) specified by   ϕCD←CC=11+exp⁡[α(PDCD←CC−πC)], where   PDCD←CC=(γC+1)⋅(1+u)+δC⋅u and   πC=1⋅2NCCNC+0⋅NCDNC. That is, this transition probability considers a $$D$$ neighbour of the centre $$C_{k,l}$$ class, and we suppose this $$D$$ neighbour has $$\gamma^{C}+1$$ neighbours in state $$C$$ (including the centre $$C_{k,l}$$ node) and $$\delta^{C}$$ in state $$D$$. Then we estimate this $$D$$ node’s utility and compare it with its typical $$C$$ neighbour (that is, the average utility $$\pi_{C}$$). We note for emphasis here that other choices could have been made in this estimate; specifically, we explicitly know that one of the $$C$$ neighbours of this $$D$$ node is the centre $$C_{k,l}$$ node. An alternative formulation of the AME system could be formed that separates out the state change of the $$D$$ neighbour in comparison with the $$C_{k,l}$$ centre (thus removing the +1 contribution) from its state changes through interaction with other $$C$$ nodes. In order to keep the equations just a little simpler, and because of the good approximation we observe in our results below, we continue to employ the simpler estimate here. We similarly estimate the Fermi function factors on the third line in equation (3.2) as   ϕCC←CD=11+exp⁡[α(PCCC←CD−πD)], where   PCCC←CD=(ηC+1)⋅1+βC⋅0 and   πD=(1+u)⋅NCDND+u⋅2NDDND. In analogy to the above argument, we consider a $$C$$ neighbour of the centre $$C_{k,l}$$ class and suppose this $$C$$ node has $$\eta^{C}+1$$ neighbours in state $$C$$ (including the centre node) and $$\beta^{C}$$ in state $$D$$. We estimate this $$C$$ node’s utility and compare it with the average utility of a $$D$$ node, $$\pi_{D}$$. The last line in the $$C_{k,l}$$ equation (3.2) describes the rewiring effect. The class $$C_{k,l+1}$$ flows to $$C_{k,l}$$ when the centre node $$C$$ drops one of its defecting neighbours and rewires randomly to another $$C$$. Similarly, the $$C_{k,l}$$ count decreases when the centre node $$C$$ drops one of its defecting neighbours and rewires randomly to another $$C$$. If the rewiring instead links to another $$D$$ node, these counts do not change. Finally, a $$C_{k-1,l}$$ node becomes $$C_{k,l}$$ if it is the recipient of a rewired edge (from another $$C$$ node), while the $$C_{k,l}$$ count decreases if a $$C_{k,l}$$ receives a rewired edge. The first group of square-bracketed terms on this line corresponds to the centre node actively rewiring to some other node in the network while the second group in square brackets describes the centre node passively receiving a rewired edge through the action of some other node in the network. Figure 4 illustrates these active and passive rewiring processes. Fig. 4. View largeDownload slide (Color online) Illustrations of nodes undergoing active and passive rewiring. (Left) Active rewiring: before the rewiring, node $$X$$ is in class $$C_{k,l}$$ (cooperator of degree $$k$$ with $$l$$ defecting neighbours). Suppose one of $$X$$’s discordant edges is selected and X actively breaks that edge shared with a defective neighbour, rewiring to a random node in the network. If node $$X$$ rewires to a node of state $$C$$ (as depicted here), then node $$X$$ moves to class $$C_{k,l-1}$$. We recall that only $$C$$ nodes actively rewire in the present model. (Right) Passive rewiring: before the rewiring, node $$X$$ is in class $$C_{k,l}$$. Suppose in the rewiring process, node $$X$$ is passively rewired through the activity of another $$C$$ node in the network. Then node $$X$$ moves to class $$C_{k+1,l}$$. Fig. 4. View largeDownload slide (Color online) Illustrations of nodes undergoing active and passive rewiring. (Left) Active rewiring: before the rewiring, node $$X$$ is in class $$C_{k,l}$$ (cooperator of degree $$k$$ with $$l$$ defecting neighbours). Suppose one of $$X$$’s discordant edges is selected and X actively breaks that edge shared with a defective neighbour, rewiring to a random node in the network. If node $$X$$ rewires to a node of state $$C$$ (as depicted here), then node $$X$$ moves to class $$C_{k,l-1}$$. We recall that only $$C$$ nodes actively rewire in the present model. (Right) Passive rewiring: before the rewiring, node $$X$$ is in class $$C_{k,l}$$. Suppose in the rewiring process, node $$X$$ is passively rewired through the activity of another $$C$$ node in the network. Then node $$X$$ moves to class $$C_{k+1,l}$$. Shifting our focus to the six flows in and out of the set depicted in the centre of the lower row of Fig. 3, we similarly obtain equation (3.3) for the $$D_{k,l}$$ compartment. The resulting system contains $$2(k_\mathrm{max} + 1)^2$$ coupled differential equations, where $$k_\mathrm{max}$$ is the maximum degree a node can have in the network. Usually, the necessary value of $$k_\mathrm{max}$$ for good accuracy depends on network structure and the mean degree. Here we choose $$k_\mathrm{max} = 50$$. We numerically solve these differential equations to semi-analytically approximate the evolution of the system, using the ode45 solver in MATLAB until the solutions reach steady state. While we consider PA and AME approaches here, we note that [70] used an approach for studying adaptive SIS networks that treated the links as the objects, classifying them according to the disease states while tracking the degree and number of infected neighbours at both end nodes. This link-based method could generally improve the accuracy compared to AME; however, it also increases the number of coupled equations from $$O(k^{2}_\mathrm{max})$$ to $$O(k^{4}_\mathrm{max})$$, where $$k_\mathrm{max}$$ is the maximum degree in the network. 4. Simulations of the CD-switching model To test the accuracy of the approximations, we study simulated dynamics on networks with $$N = 1,000$$ nodes and $$M = 5,000$$ edges. That is, the mean degree of the network is fixed to be $$\langle k \rangle = 2M/N = 10$$. We set the parameter $$\alpha=30$$ in the Fermi function used for imitating strategies. We study dynamics for different initial fractions of defectors, denoted by $$\rho$$. We consider $$\rho = 0.5$$ first, and explore the influence of this parameter later. We draw the initial networks from an Erdős–Rényi $$G(N, M)$$ random graph model of $$N$$ nodes and $$M$$ edges distributed uniformly and independently between the nodes. For large $$N$$, the initial degree distribution is approximated by the Poisson distribution of mean $$\langle k \rangle$$, with the probability of a selected node having degree $$k$$ being $$p_{k} = {\langle k \rangle^{k}e^{-\langle k \rangle}}/{k!}.$$ In the present model, updating only discordant edges (i.e. $$CD$$), the dynamics stop when there are no discordant edges remaining. Unless stated otherwise, we perform $$1,000$$ simulations for each condition and report the average. 4.1. Final level of cooperation To study the effect of the cost-benefit ratio $$u$$ and strategy update rate $$w$$, we focus on the level of cooperation obtained in the final (stationary) states of the model, starting from an initial fraction of defectors $$\rho=0.5$$. In Fig. 5, we present heat maps generated by simulations and AME approximations of the final fractions of cooperating nodes and CC links. As qualitatively expected, the final fraction of cooperating nodes is higher when the possible additional payoff for defecting, $$u$$, is smaller. When $$w$$ is close to $$0$$, we observe a moderate level of cooperators, as expected because the discordant edges can resolve themselves by rewiring with fewer strategy updates. Away from the small values of $$u$$ or $$w$$, the final network states are dominated by defectors. The AME approximation generally captures these high- and low-cooperation regions of the $$(u,w)$$ parameter space, up to a modest displacement in the precise position of the phase transition. Fig. 5. View largeDownload slide (Color online) Results from simulations (left column) and approximate master equations (AME) (right column) of the final fractions of cooperators (top row) and CC edges (bottom row) for different combinations of the cost-benefit ratio $$u$$ and strategy updating probability $$w$$. The initial fraction of defectors is $$\rho=0.5$$. For both the $$u$$ and $$w$$ axes, we use steps of 0.05 and plot the results from stationary states. Simulation results here are averaged over 50 realizations at each parameter set. These visualizations were generated from results on a regular grid through bilinear interpolation, leading to some clearly apparent grid artefacts. While some discrepancies between simulation and AME results are clearly present, we note in particular that the position of the phase transition in the $$(u,w)$$ parameter space is well approximated by the AME system. Fig. 5. View largeDownload slide (Color online) Results from simulations (left column) and approximate master equations (AME) (right column) of the final fractions of cooperators (top row) and CC edges (bottom row) for different combinations of the cost-benefit ratio $$u$$ and strategy updating probability $$w$$. The initial fraction of defectors is $$\rho=0.5$$. For both the $$u$$ and $$w$$ axes, we use steps of 0.05 and plot the results from stationary states. Simulation results here are averaged over 50 realizations at each parameter set. These visualizations were generated from results on a regular grid through bilinear interpolation, leading to some clearly apparent grid artefacts. While some discrepancies between simulation and AME results are clearly present, we note in particular that the position of the phase transition in the $$(u,w)$$ parameter space is well approximated by the AME system. In Fig. 6, we further explore the final level of cooperation and assess the accuracies of the two semi-analytical approximations. In the left panel of Fig. 6, we consider $$w = 0$$, $$w = 0.05$$, $$w = 0.1$$ and $$w = 0.5$$, plotting the final fraction of nodes in state $$C$$ versus the cost-benefit ratio, $$u$$. There are three sets of outcomes here: simulations (markers), PA (dotted lines) and AME (dashed lines). For $$w = 0$$, there are no strategy updates and the final fraction of $$C$$ nodes is fixed at $$1-\rho$$. For $$w > 0$$, strategy updates come into play with larger values of the cost-benefit ratio $$u$$ driving stronger incentive for nodes to defect, decreasing the final fraction of cooperators. As seen in the figure, the cases $$w = 0.1$$ and $$w = 0.5$$ include higher levels of cooperation at small $$u$$ that decreases slightly as $$u$$ increases before suddenly dropping to zero near $$u \doteq 0.2$$ and $$u \doteq 0.6$$, respectively. This phenomenon is qualitatively captured by both PA and AME, though the quantitative description from AME is much more accurate, particularly in terms of these critical points. In contrast, the PA prediction of the fraction of cooperators drops to zero at much smaller values of $$u$$. Similarly, in the case $$w = 0.05$$, the level of cooperation in the simulation results decreases with increasing $$u$$, but without the sharp drop-off; in contrast to both the simulations and AME, the PA result drops to zero around $$u \doteq 0.9$$. Fig. 6. View largeDownload slide (Color online) (Left) Final fraction of cooperators versus cost-benefit ratio $$u$$ for different strategy updating probabilities $$w$$. Markers are averages from 1,000 simulations, dotted lines are pair approximation (PA) results, and dashed lines are from approximate master equations (AME). The PA results are different from those shown in [19], but we have confirmed the accuracy of our results by personal communication with those authors. (Right) Final fraction of cooperators versus $$w$$ for different cost-benefit ratio $$u$$. Markers, dotted lines, and dashed lines again indicate simulations, PA, and AME, respectively. In both panels, we observe that AME is typically more accurate than PA, especially so near the phase transition where the fraction of cooperators goes to zero. Fig. 6. View largeDownload slide (Color online) (Left) Final fraction of cooperators versus cost-benefit ratio $$u$$ for different strategy updating probabilities $$w$$. Markers are averages from 1,000 simulations, dotted lines are pair approximation (PA) results, and dashed lines are from approximate master equations (AME). The PA results are different from those shown in [19], but we have confirmed the accuracy of our results by personal communication with those authors. (Right) Final fraction of cooperators versus $$w$$ for different cost-benefit ratio $$u$$. Markers, dotted lines, and dashed lines again indicate simulations, PA, and AME, respectively. In both panels, we observe that AME is typically more accurate than PA, especially so near the phase transition where the fraction of cooperators goes to zero. Similarly, in the right panel of Fig. 6 we plot the final fraction of cooperators $$C$$ versus $$w$$ for different cost-benefit ratios, $$u = 0$$, $$u = 0.01$$, $$u = 0.2$$ and $$u = 0.8$$. For $$w=0$$, no nodes update their strategies and the final fraction of $$C$$ nodes is $$1-\rho=0.5$$. Larger $$w$$ increases both the number of strategy updates and their overall effect on the final state. In the cases $$u = 0$$ and $$u = 0.01$$, $$u$$ is so small that there is almost no incentive to defect, so in the strategy updating consideration $$C$$’s and $$D$$’s are, holding everything else equal, almost indistinguishable. However, in rewiring an edge the $$C$$ node drops its $$D$$ neighbour, reducing the number of partners for $$D$$, and over time the $$D$$ nodes experience lower payoffs because the numbers of their neighbours decrease. This decrease in degree results in lower total payoff, increasing the probability of a $$D$$ node changing to a $$C$$ strategy, leading to the dominance of $$C$$’s for small $$u$$. In contrast, for $$u = 0.8$$, the $$C$$ nodes become more disadvantaged with increased $$w$$ since the payoff of defecting is high, and the final fraction of cooperators decreases to zero around $$w \doteq 0.1$$. The case $$u = 0.2$$ is intermediate between the competing effects of these two extremes. The final fraction of cooperators increases for values of $$w$$ that are not too large due to the above-described rewiring effect. But sufficiently large strategy update rates (i.e. fewer rewiring steps) combined with an apparently sufficient incentive to defect leads to final network states dominated by defectors. In the simulations, this transition for $$u=0.2$$ appears to occur near $$w\doteq 0.4$$. While the PA and AME results qualitatively describe all of these effects well, AME does a better job capturing this transition point. Taking these results together, we note that in general the increased frequency of breaking and rewiring $$CD$$ bonds (i.e. smaller $$w$$) can directly support cooperation, in agreement with [36, 37], as observed here both in terms of the apparent phase transition and the behaviour at large $$u$$ in the right panel of Fig. 6. But we also note that the results include situations where increasing this frequency (decreasing $$w$$) decreases the total level of cooperation, because of the complex interplay in the model dynamics. 4.2. Network dynamics Having explored the populations in the final states, we consider how the networks evolve to those states. Specifically, we investigate the evolution of five fundamental quantities of the networks: the fractions of nodes in states $$C$$ and $$D$$, and the fractions of edges that are $$CC$$, $$CD$$ and $$DD$$. We recall that the total numbers of nodes and edges are constant during the dynamics, requiring both $$C+D$$ fractions and $$CC+CD+DD$$ fractions to be equal to $$1$$. We visualize the average trajectories through this phase space for a collection of different $$(u,w)$$ parameters in Fig. 7, starting from an initial fraction of defectors $$\rho=0.5$$ distributed across an Erdős–Rényi $$G(N, M)$$ random graph, comparing simulations with PA and AME predictions. As such, the initial fraction of $$CC$$, $$DD$$ and $$CD$$ edges is $$0.25$$, $$0.25$$ and $$0.5$$, respectively; that is, all networks start at $$C=0.5$$, $$CC=0.25$$, $$CD=0.5$$. As the networks evolve, these fractions change, tracing out trajectories in the $$(C,CD)$$ and $$(CC,CD)$$ coordinates in the figure. We perform 50 simulations at each set of parameter values and compute the averages across simulations at each time step. The networks evolve until there are no discordant edges (i.e. $$CD = 0$$). Fig. 7. View largeDownload slide (Color online) Phase space visualizations of the temporal evolution of the fraction of edges that are $$CD$$ versus the fraction of nodes that are $$C$$ (left column) and versus the fraction of edges that are $$CC$$ (right column) for different values of the cost-benefit ratio $$u$$ and strategy updating probability $$w$$ (rows): (a and b) $$u = 0.2$$, $$w = 0.1$$, (c and d) $$u = 0.2$$, $$w = 0.3$$, (e and f) $$u = 0.5$$, $$w = 0.1$$, and (g and h) $$u = 0.5$$, $$w = 0.3$$. Solid lines are the averages of 50 simulations results, dash-dot lines are the pair approximation (PA), and dashed lines are the semi-analytical results of approximate master equations (AME). All networks start at $$C=0.5$$, $$CC=0.25$$ and $$CD=0.5$$. As the system evolves, on average the number of $$CD$$ decreases over time until none of these discordant edges remain ($$CD=0$$). Sufficiently far from the phase transition (as in the top and bottom rows), both approximations are reasonably good. Near the phase transition there are ranges of parameters where only $$AME$$ provides a good description (such as in the second row, where $$PA$$ incorrectly predicts the position relative to the phase transition; compare with the markers and lines in the right panel of Fig. 6). Also near the phase transition, there are other parameters where neither approximation captures the qualitative behaviour (as in the third row, with $$u=0.5$$ and $$w=0.1$$; compare with the location of the observed and predicted phase transition in Fig. 5 and with the markers and lines in the left panel of Fig. 6). Fig. 7. View largeDownload slide (Color online) Phase space visualizations of the temporal evolution of the fraction of edges that are $$CD$$ versus the fraction of nodes that are $$C$$ (left column) and versus the fraction of edges that are $$CC$$ (right column) for different values of the cost-benefit ratio $$u$$ and strategy updating probability $$w$$ (rows): (a and b) $$u = 0.2$$, $$w = 0.1$$, (c and d) $$u = 0.2$$, $$w = 0.3$$, (e and f) $$u = 0.5$$, $$w = 0.1$$, and (g and h) $$u = 0.5$$, $$w = 0.3$$. Solid lines are the averages of 50 simulations results, dash-dot lines are the pair approximation (PA), and dashed lines are the semi-analytical results of approximate master equations (AME). All networks start at $$C=0.5$$, $$CC=0.25$$ and $$CD=0.5$$. As the system evolves, on average the number of $$CD$$ decreases over time until none of these discordant edges remain ($$CD=0$$). Sufficiently far from the phase transition (as in the top and bottom rows), both approximations are reasonably good. Near the phase transition there are ranges of parameters where only $$AME$$ provides a good description (such as in the second row, where $$PA$$ incorrectly predicts the position relative to the phase transition; compare with the markers and lines in the right panel of Fig. 6). Also near the phase transition, there are other parameters where neither approximation captures the qualitative behaviour (as in the third row, with $$u=0.5$$ and $$w=0.1$$; compare with the location of the observed and predicted phase transition in Fig. 5 and with the markers and lines in the left panel of Fig. 6). For comparison, we include both PA and AME in the figure. While some features of the trajectories are better captured than others, both methods in general do reasonably well qualitatively, with better overall accuracy for AME, as seen in the figure. We note that both approximations are qualitatively incorrect for the case $$u=0.5$$, $$w=0.1$$. Recalling the simulation and AME results in the Fig. 5 phase diagram, we note that this parameter set is within the relatively narrow range between the observed location of the phase transition and the AME prediction. 4.3. Degree distributions The AME method utilizes more information and has higher computational cost than PA, but as we have seen it generally provides a better approximation for the current model. Moreover, and unlike PA, AME explicitly includes information about degree distributions. In Fig. 8, we plot the final state degree distributions of $$C$$ and $$D$$ nodes, comparing the simulations and AME predictions. In each case, since we initialize with random strategies on the Erdős–Rényi $$G(N, M)$$ random graph model, the initial degree distributions are approximately Poisson (for large $$N$$). We examine degree distributions in the final states, revisiting the same parameters considered previously: $$u = 0.2, w = 0.1$$; $$u = 0.2, w = 0.3$$; $$u = 0.5, w = 0.1$$; and $$u = 0.5, w = 0.3$$. In the case $$u = 0.5, w = 0.3$$, it is perhaps not surprisingly that AME gives an excellent prediction of the final degree distribution, since we observed excellent prediction of the trajectory of the dynamics in this case above (Fig. 7). In the final state of this case, $$D$$ nodes dominate the whole network, with no $$C$$ nodes left. Fig. 8. View largeDownload slide (Color online) Degree distributions in the stationary states for different values of the cost-benefit ratio $$u$$ and strategy updating probability $$w$$: (a) $$u = 0.2$$, $$w = 0.1$$; (b) $$u = 0.2$$, $$w = 0.3$$; (c) $$u = 0.5$$, $$w = 0.1$$; and (d) $$u = 0.5$$, $$w = 0.3$$. Bars indicate averages from 50 simulations, with colours distinguishing the degree distributions of cooperator and defector nodes. Lines indicate the prediction from the semi-analytical approximate master equations (AME). We note the qualitatively good agreement of the AME prediction with simulations, except in case (c) where AME predicts extinction of the cooperator nodes, as we have previously seen (see Fig. 6(e and f)). Fig. 8. View largeDownload slide (Color online) Degree distributions in the stationary states for different values of the cost-benefit ratio $$u$$ and strategy updating probability $$w$$: (a) $$u = 0.2$$, $$w = 0.1$$; (b) $$u = 0.2$$, $$w = 0.3$$; (c) $$u = 0.5$$, $$w = 0.1$$; and (d) $$u = 0.5$$, $$w = 0.3$$. Bars indicate averages from 50 simulations, with colours distinguishing the degree distributions of cooperator and defector nodes. Lines indicate the prediction from the semi-analytical approximate master equations (AME). We note the qualitatively good agreement of the AME prediction with simulations, except in case (c) where AME predicts extinction of the cooperator nodes, as we have previously seen (see Fig. 6(e and f)). In the other three cases, although the AME predictions are not as accurate, we can still see the AME provides a qualitative picture of the final degree distribution. Once again, the worst case of the four parameter sets visualized here is the $$u=0.5$$, $$w=0.1$$, where the AME prediction is on the wrong side of the phase transition. 4.4. The effect of the initial fraction of defectors, $$\rho$$ In Fig. 9, we explore the effect of the initial fraction of defectors, $$\rho$$. In this figure, we fix the parameter to be $$w = 0.1$$ in the left panel and $$u = 0.2$$ in the right panel, changing the other parameter ($$u$$ and $$w$$, respectively) to view the final fraction of cooperators against $$\rho$$ in stationary states, comparing simulation results (markers) with our AME approximation (lines). By construction, all results in these panels should connect the upper left and lower right corners because when $$\rho = 0$$ (respectively, $$\rho = 1$$) there are no $$D$$ ($$C$$) nodes, no discordant edges, and no dynamics. Fig. 9. View largeDownload slide (Color online) (Left) Fraction of cooperators remaining in the stationary state versus the initial fraction of defectors $$\rho$$, for strategy updating probability $$w = 0.1$$ and various cost-benefit ratios $$u$$: 0, 0.1, ..., 1. (Right) Fraction of cooperators versus $$\rho$$ for $$u = 0.2$$ and various strategy updating probabilities $$w$$: 0, 0.1, ..., 1. In both figures, markers are the averages from 50 simulations and the lines are the semi-analytical predictions from approximate master equations (AME). Fig. 9. View largeDownload slide (Color online) (Left) Fraction of cooperators remaining in the stationary state versus the initial fraction of defectors $$\rho$$, for strategy updating probability $$w = 0.1$$ and various cost-benefit ratios $$u$$: 0, 0.1, ..., 1. (Right) Fraction of cooperators versus $$\rho$$ for $$u = 0.2$$ and various strategy updating probabilities $$w$$: 0, 0.1, ..., 1. In both figures, markers are the averages from 50 simulations and the lines are the semi-analytical predictions from approximate master equations (AME). In the left panel, for the different $$u$$ curves, the final fraction of cooperators first gradually decreases and then drops to zero suddenly at different values of $$\rho$$. AME predicts this qualitative behaviour for different levels of $$u$$. Moreover, fixing $$w = 0.1$$, smaller $$u$$ results in higher cooperation, since the incentive of defecting is less when $$u$$ is small. In the right panel, fixing $$u = 0.2$$, the diagonal line is the case $$w = 0$$, where there are no strategy updates and the cooperation level remains the same during the evolution. For small and moderate $$w$$, the curves decrease with increasing $$\rho$$ (though more quickly for smaller $$w$$), followed by a sharp drop to zero at decreasing values of $$\rho$$ for increased $$w$$. When $$w$$ is large (greater rates of strategy updates), the final cooperative level drops to zero for all non-zero $$\rho$$. As we show in this panel, the AME curves suddenly drop to zero when $$w \geq 0.4$$. 5. A model variant with $$\boldsymbol{DD}$$ rewiring In this section, we study a slightly different variant of the partner switching evolution game model, also introduced by [19], where $$DD$$ edges can also rewire as defector nodes seek out new partners that they might better exploit. We are particularly interested in how this additional rewiring affects the final cooperation level. In this variant, each time step starts by uniformly at random selecting from the union of $$CD$$ and $$DD$$ edges. (Recall that only $$CD$$ edges were selected in the original model above.) If a $$CD$$ edge is picked, the dynamics proceed as in the original model: with probability $$w$$, the strategy update process happens; otherwise (that is, with probability $$1-w$$) the $$C$$ node along this edge unilaterally drops the partnership with its $$D$$ neighbour and rewires to another node. In contrast, if a $$DD$$ edge is picked, then with probability $$w$$ nothing happens (there is no strategy update); otherwise (that is, with probability $$1-w$$) one of the two $$D$$ ends, selected with equal probability, drops the connection and rewires uniformly at random to another node in the network to whom it is not already connected. We note that the total numbers of nodes, $$N$$, and edges, $$M$$, remain constant, as in the original model. This variant model stops evolving when there are no $$CD$$ or $$DD$$ edges remaining in the system; that is, all edges in a final, frozen state are $$CC$$. However, this does not require that all $$D$$ nodes are removed from the system, only that they are each left isolated with no connections. Another possibility, in which the model variant as stated above never reaches a stopping condition, is that the system evolves to a statistically stationary state where all $$C$$ nodes have been removed so that all edges are $$DD$$ edges and the continued rewiring of these $$DD$$ edges simply serves to repeatedly randomize the network of defector nodes. Since in this case there are no $$C$$ nodes left to emulate in a strategy update nor to connect to in a rewiring, we do not consider the dynamics further beyond this event. Both of these cases are visualized in Fig. 10. In the left panel, the $$C$$ nodes own all of the edges, having isolated the exploitation efforts of the $$D$$ nodes. In the right panel, only $$D$$ nodes remain and all edges are $$DD$$. Fig. 10. View largeDownload slide (Color online) Visualization of stationary states of the model variant with $$DD$$ rewiring, for $$N \,{=}\, 1,000$$ nodes, $$M \,{=}\, 5,000$$ edges, and initial fraction of defectors $$\rho \,{=}\, 0.5$$, for different parameters. Colours correspond to the two node states: cooperating (light/blue) and defecting (dark/red). (Left) For cost-benefit ratio $$u \,{=}\, 0.5$$ and strategy updating probability $$w \,{=}\, 0.1$$, $$D$$ nodes remain in the system but they have each been isolated with zero degree. (Right) For $$u \,{=}\, 1$$ and $$w \,{=}\, 0.5$$, the system evolves until only $$D$$ nodes exist in the network and all the edges are $$DD$$ edges. These visualizations were created using the Yifan Hu layout in Gephi. Fig. 10. View largeDownload slide (Color online) Visualization of stationary states of the model variant with $$DD$$ rewiring, for $$N \,{=}\, 1,000$$ nodes, $$M \,{=}\, 5,000$$ edges, and initial fraction of defectors $$\rho \,{=}\, 0.5$$, for different parameters. Colours correspond to the two node states: cooperating (light/blue) and defecting (dark/red). (Left) For cost-benefit ratio $$u \,{=}\, 0.5$$ and strategy updating probability $$w \,{=}\, 0.1$$, $$D$$ nodes remain in the system but they have each been isolated with zero degree. (Right) For $$u \,{=}\, 1$$ and $$w \,{=}\, 0.5$$, the system evolves until only $$D$$ nodes exist in the network and all the edges are $$DD$$ edges. These visualizations were created using the Yifan Hu layout in Gephi. 5.1. Semi-analytical methods of approximation As before, we study this model variant with a combination of simulations and approximate analytic models, comparing the PA and AME approaches with simulation results. Following the notation of the original model in Section 3, the PA equations for this variant become   dNCdt= w⋅NCD⋅tanh⁡[α2(π¯C−π¯D)]dNCCdt= w⋅(NCDϕC→D−2NCDNCCNCϕD→C+NCDNCDNDϕC→D) +(1−w)⋅NCNNCDdNDDdt= w⋅(NCDϕD→C−2NCDNDDNDϕC→D+NCDNCDNCϕD→C) −(1−w)⋅NCNNDD (5.1) where we note the only difference between equation (5.1) and equation (3.1) is the appearance of the last term in the $$dN_{DD}/dt$$ equation. This term captures the possibility in this variant model for a $$D$$ node to dismiss its defective partner and rewire to a $$C$$ node, thus decreasing the $$DD$$ count. Similarly, we modify the AME equations to account for the rewiring of $$DD$$ edges, yielding   dCk,ldt =w{ϕk,lD⋅(k−l)Dk,l−ϕk,lC⋅lCk,l +ϕCD←CC⋅γC(l+1)Ck,l+1−ϕCD←CC⋅γClCk,l +ϕCC←CD⋅βC(k−l+1)Ck,l−1−ϕCC←CD⋅βC(k−l)Ck,l} +(1−w){NCN[(l+1)Ck,l+1−lCk,l]+NCDN[Ck−1,l−Ck,l] +NDDN[Ck−1,l−1−Ck,l]} (5.2)  dDk,ldt =w{−ϕk,lD⋅(k−l)Dk,l+ϕk,lC⋅lCk,l +ϕDD←DC⋅γD(l+1)Dk,l+1−ϕDD←DC⋅γDlDk,l +ϕDC←DD⋅βD(k−l+1)Dk,l−1−ϕDC←DD⋅βD(k−l)Dk,l} +(1−w){[(k−l+1)Dk+1,l−(k−l)Dk,l]+NCDN[Dk−1,l−Dk,l] +NC2N[(l+1)Dk,l+1−lDk,l]+NDDN[Dk−1,l−1−Dk,l] +12[(l+1)Dk+1,l+1−lDk,l]}. (5.3) Again, the additional terms here compared to equations (3.2) and (3.3) account for the rewiring of $$DD$$ edges, accounting for the rates of $$D$$ nodes abandoning $$D$$ neighbours and rewiring to $$C$$ nodes, $$D$$ nodes passively gaining new connections through this rewiring, and $$D$$ nodes being abandoned by their $$D$$ neighbours. Note the $$1/2$$ factors in the last two lines of the $$D_{k,l}$$ equation, accounting for an individual node on a $$DD$$ keeping and losing the edge. 5.2. Final level of cooperation As before, we compare the final fraction of cooperators in the simulations versus the PA and AME predictions. In the left panel of Fig. 11, we consider $$w = 0$$, $$w = 0.05$$, $$w = 0.1$$, and $$w = 0.5$$, plotting the fraction of $$C$$ nodes in the final states versus the cost-benefit ratio, $$u$$. In the right panel, we fix values of $$u$$ and vary $$w$$. Again, markers indicate simulation results, dotted lines are from the PA equations, and dashed lines are from AME. Fig. 11. View largeDownload slide (Color online) (Left) Fraction of cooperators remaining in the stationary state of the model variant with $$DD$$ rewiring versus cost-benefit ratio $$u$$, for different strategy updating probabilities $$w$$. Markers are averages from 1,000 simulations, dotted lines are the semi-analytical results from pair approximation (PA), and the dashed lines are the semi-analytical results from approximate master equations (AME). (Right) Fraction of cooperators remaining in the stationary state versus $$w$$ for different $$u$$ values, with markers and lines as in the left panel. Fig. 11. View largeDownload slide (Color online) (Left) Fraction of cooperators remaining in the stationary state of the model variant with $$DD$$ rewiring versus cost-benefit ratio $$u$$, for different strategy updating probabilities $$w$$. Markers are averages from 1,000 simulations, dotted lines are the semi-analytical results from pair approximation (PA), and the dashed lines are the semi-analytical results from approximate master equations (AME). (Right) Fraction of cooperators remaining in the stationary state versus $$w$$ for different $$u$$ values, with markers and lines as in the left panel. Comparing Fig. 11 for this variant with the results in Fig. 6, the inclusion of $$DD$$ rewiring here leads to slightly larger final fractions of $$C$$ nodes for small $$w>0$$. In particular, whereas an existing $$DD$$ link in the original model can only be changed if the state of one of the two $$D$$ nodes changes in a strategy update, the $$DD$$ rewiring in this variant increases the numbers of $$CD$$ edges and thus leaves open the possibility of $$C$$ nodes becoming more favourable because of their yet higher degree. However, for larger values of $$w$$, the more rapid rate of strategy updating leads to defectors dominating the system before the cooperators can gain any advantage from increased degrees. For further comparison, we consider the final fractions of $$C$$ nodes and $$CC$$ edges for different combinations of $$u$$ and $$w$$ in Fig. 12, as compared with the similar plots in Fig. 5. While many of the general features remain the same in the presence of this DD-rewiring variant, a notable difference is in the final fraction of $$CC$$ edges for small $$w$$. In particular, at $$w=0$$ (no strategy updates), the original model results visualized in Fig. 5 limit to a final $$CC$$ level of 0.75, since the initial $$DD$$ edges (at 0.25 for $$\rho=0.5$$) cannot ever rewire under these settings. But these edges of course do rewire in the $$DD$$-rewiring variant, leading $$CC$$ to limit to 1 as $$w\to 0$$. Fig. 12. View largeDownload slide (Color online) Results from simulations (left column) and approximate master equations (AME) (right column) of the final fractions of $$C$$ nodes (top row) and $$CC$$ links (bottom row) for different combinations of the cost-benefit ratio $$u$$ and strategy updating probability $$w$$ in the model variant with $$DD$$ rewiring. The initial fraction of defectors is $$\rho=0.5$$. For both the $$u$$ and $$w$$ axes, we use steps of 0.05 and plot the results from stationary states. Simulation results here are averaged over 50 realizations at each parameter set. These visualizations were generated from results on a regular grid through bilinear interpolation, leading to some clearly apparent grid artefacts. As in the original model without $$DD$$ rewiring (Fig. 5), while some discrepancies between simulation and AME results are clearly present, we note in particular that the position of the phase transition in the $$(u,w)$$ parameter space is well approximated by the AME system. Fig. 12. View largeDownload slide (Color online) Results from simulations (left column) and approximate master equations (AME) (right column) of the final fractions of $$C$$ nodes (top row) and $$CC$$ links (bottom row) for different combinations of the cost-benefit ratio $$u$$ and strategy updating probability $$w$$ in the model variant with $$DD$$ rewiring. The initial fraction of defectors is $$\rho=0.5$$. For both the $$u$$ and $$w$$ axes, we use steps of 0.05 and plot the results from stationary states. Simulation results here are averaged over 50 realizations at each parameter set. These visualizations were generated from results on a regular grid through bilinear interpolation, leading to some clearly apparent grid artefacts. As in the original model without $$DD$$ rewiring (Fig. 5), while some discrepancies between simulation and AME results are clearly present, we note in particular that the position of the phase transition in the $$(u,w)$$ parameter space is well approximated by the AME system. 6. Conclusion We have investigated a node-based prisoner’s dilemma game played on a network coevolving with player strategy updates. Our study includes two model variants for rewiring: one where only $$CD$$ edges can rewire, by $$C$$ nodes dropping a link to a $$D$$ neighbour in favour of a new partner; and a variant where $$DD$$ edges can also rewire. We explore the parameter space to investigate the competing effects of strategy updates and partner switching, as well as the initial levels of cooperation versus defection. We compare our simulations to the existing PA developed in [19], and we develop an approximation using AME to more accurately capture the transitions between the properties of the final states of these models. We also use the AME method to estimate the final-state degree distributions for different parameters. We are particularly interested in the features that determine the final level of cooperation and overall utility in the network. Revisiting Fig. 6, and the corresponding results for the model variant in Fig. 11, it is clear that the most effective way to increase the final fraction of cooperators is to directly reduce the cost-benefit ratio, $$u$$, so that players do not have as much incentive to defect. Alternatively, if the strategy update rate $$w$$ is small enough, the effects of rewiring give greater advantages to $$C$$ nodes as they accumulate larger numbers of playing partners, which then results in larger numbers of cooperators through the strategy updates. Of course, one can also directly decrease the initial fraction of defectors. The question of how to maximize the total utility or payoff relates to the types of edges. In the partner-switching evolutionary game model we study here, the total number of nodes and edges remain fixed, no matter how the networks and node states change. Utility comes from every edge in the network, with each $$CC$$, $$CD$$ and $$DD$$ edge contributing (from the two nodes) $$2$$, $$1+u$$, and $$2u$$, respectively, where $$u \in (0,1)$$. With no $$CD$$ edges in the final state, maximizing the final overall utility reduces to maximizing the number of $$CC$$ links. In particular, comparing the right panels of Figs 5 and 12 for the two model variants, whether or not $$DD$$ edges can rewire significantly affects the $$CC$$ count (and thus the total system utility) for small strategy update rates. Acknowledgments We are grateful to the reviewers for identifying multiple important references cited here. Funding Research reported in this publication was supported by the Eunice Kennedy Shriver National Institute of Child Health and Human Development of the National Institutes of Health under Award Number R01HD075712. The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institutes of Health. References 1. Gibbons, R. (1992) A Primer in Game Theory . Coleshill, Birmingham B46 1EE, UK: Harvester Wheatsheaf. 2. Harsanyi, J. C., Selten, R. (1988) A General Theory of Equilibrium Selection in Games , vol. 1. Cambridge, MA 02142, USA: MIT Press Books. 3. Myerson, R. B. (1991) Game Theory: Analysis of Conflict . Cambridge, MA: Harvard University Press. 4. Hauert, C., Holmes, M. & Doebeli, M. (2006) Evolutionary games and population dynamics: maintenance of cooperation in public goods games. Proc. R. Soc. Lond B Biol. Sci. , 273, 2565– 2571. Google Scholar CrossRef Search ADS   5. Hofbauer, J. & Sigmund, K. (2003) Evolutionary game dynamics. Bull. Am. Math. Soc. , 40, (4), 479– 519. Google Scholar CrossRef Search ADS   6. Nowak, M. A. & May, R. M. (1993) The spatial dilemmas of evolution. Int. J. Bifurcat. Chaos , 3, (01), 35– 78. Google Scholar CrossRef Search ADS   7. Smith, J. M. (1982) Evolution and the Theory of Games . Cambridge CB2 8BS, UK: Cambridge University Press. 8. Weibull, J. W. (1997) Evolutionary Game Theory . Cambridge, MA 02142, USA: MIT Press. 9. Mitteldorf, J. & Wilson, D. S. (2000) Population viscosity and the evolution of altruism. J. Theoret. Biol. , 204, (4), 481– 496. Google Scholar CrossRef Search ADS   10. Fletcher, J. A. & Zwick, M. (2007) The evolution of altruism: game theory in multilevel selection and inclusive fitness. J. Theoret. Biol. , 245, (1), 26– 36. Google Scholar CrossRef Search ADS   11. Axelrod, R. (2006) The Evolution of Cooperation , revised edn. New York, NY 10104, USA: Basic Books. 12. Miller, J. H. (1996) The coevolution of automata in the repeated prisoner’s dilemma. J. Econ. Behav. Organ. , 29, (1), 87– 112. Google Scholar CrossRef Search ADS   13. Mertens, J.-F. (1989) Repeated Games . New York, NY 10036, USA: Springer. 14. Fowler, J. H. & Laver, M. (2008) A tournament of party decision rules. J. Conflict Resolut. , 52, (1), 68– 92. Google Scholar CrossRef Search ADS   15. Nisan, N., Roughgarden, T., Tardos, E. & Vazirani, V. V. (2007) Algorithmic Game Theory , vol. 1. Cambridge: Cambridge University Press. 16. Abramson, G. & Kuperman, M. (2001) Social games in a social network. Phys. Rev. E , 63, (3), 030901. Google Scholar CrossRef Search ADS   17. Ebel, H. & Bornholdt, S. (2002) Coevolutionary games on networks. Phys. Rev. E , 66, (5), 056118. Google Scholar CrossRef Search ADS   18. Fu, F., Hauert, C., Nowak, M. A. & Wang, L. (2008) Reputation-based partner choice promotes cooperation in social networks. Phys. Rev. E , 78, (2), 026117. Google Scholar CrossRef Search ADS   19. Fu, F., Wu, T. & Wang, L. (2009) Partner switching stabilizes cooperation in coevolutionary prisoner’s dilemma. Phys. Rev. E , 79, (3), 036101. Google Scholar CrossRef Search ADS   20. Hauert, C. & Doebeli, M. (2004) Spatial structure often inhibits the evolution of cooperation in the snowdrift game. Nature , 428, (6983), 643– 646. Google Scholar CrossRef Search ADS   21. Hidalgo, J., Grilli, J., Suweis, S., Maritan, A. & Muñoz, M. A. (2015) Cooperation, competition and the emergence of criticality in communities of adaptive systems. J. Stat. Mech.: Theor. Experi.,  2016, (3), 033203. 22. Jackson, M. O. & Watts, A. (2002) On the formation of interaction networks in social coordination games. Games Econ. Behav. , 41, (2), 265– 291. Google Scholar CrossRef Search ADS   23. Killingback, T. & Doebeli, M. (1996) Spatial evolutionary game theory: Hawks and Doves revisited. Proc. R. Soc. Lond. B Biol. Sci. , 263, (1374), 1135– 1144. Google Scholar CrossRef Search ADS   24. Lieberman, E., Hauert, C. & Nowak, M. A. (2005) Evolutionary dynamics on graphs. Nature , 433, (7023), 312– 316. Google Scholar CrossRef Search ADS   25. Nowak, M. A. & May, R. M. (1992) Evolutionary games and spatial chaos. Nature , 359, (6398), 826– 829. Google Scholar CrossRef Search ADS   26. Kerr, B., Riley, M. A., Feldman, M. W. & Bohannan, B. J. (2002) Local dispersal promotes biodiversity in a real-life game of rock–paper–scissors. Nature , 418, (6894), 171– 174. Google Scholar CrossRef Search ADS   27. Pinheiro, F. L., Santos, F. C. & Pacheco, J. M. (2016) Linking individual and collective behavior in adaptive social networks. Phys. Rev. Lett. , 116, (12), 128702. Google Scholar CrossRef Search ADS   28. Santos, F. C., Pacheco, J. M. & Lenaerts, T. (2006) Cooperation prevails when individuals adjust their social ties. PLoS Comput. Biol. , 2, (10), e140. Google Scholar CrossRef Search ADS   29. Hauert, C., De Monte, S., Hofbauer, J. & Sigmund, K. (2002) Replicator dynamics for optional public good games. J. Theoret. Biol. , 218, (2), 187– 194. Google Scholar CrossRef Search ADS   30. Perc, M., Gómez-Gardeñes, J., Szolnoki, A., Floría, L. M. & Moreno, Y. (2013) Evolutionary dynamics of group interactions on structured populations: a review. J. R. Soc. Interface , 10, (80), 20120997. Google Scholar CrossRef Search ADS   31. Szolnoki, A., Perc, M. & Szabó, G. (2009) Topology-independent impact of noise on cooperation in spatial public goods games. Phys. Rev. E , 80, (5), 056109. Google Scholar CrossRef Search ADS   32. Pacheco, J. M., Traulsen, A. & Nowak, M. A. (2006) Active linking in evolutionary games. J. Theoret. Biol. , 243, (3), 437– 443. Google Scholar CrossRef Search ADS   33. Pacheco, J. M., Traulsen, A. & Nowak, M. A. (2006) Coevolution of strategy and structure in complex networks with dynamical linking. Phys. Rev. Lett. , 97, (25), 258103. Google Scholar CrossRef Search ADS   34. Perc, M. & Szolnoki, A. (2010) Coevolutionary games—a mini review. BioSystems , 99, (2), 109– 125. Google Scholar CrossRef Search ADS   35. Szolnoki, A. & Perc, M. (2009) Emergence of multilevel selection in the prisoner’s dilemma game on coevolving random networks. New J. Phys. , 11, (9), 093033. Google Scholar CrossRef Search ADS   36. Szolnoki, A. & Perc, M. (2009) Resolving social dilemmas on evolving random networks. Europhys. Lett. , 86, (3), 30007. Google Scholar CrossRef Search ADS   37. Szolnoki, A., Perc, M. & Danku, Z. (2008) Making new connections towards cooperation in the prisoner’s dilemma game. Europhys. Lett. , 84, (5), 50007. Google Scholar CrossRef Search ADS   38. Gómez-Gardeñes, J., Campillo, M., Floría, L. & Moreno, Y. (2007) Dynamical organization of cooperation in complex topologies. Phys. Rev. Lett. , 98, (10), 108103. Google Scholar CrossRef Search ADS   39. Perc, M. (2009) Evolution of cooperation on scale-free networks subject to error and attack. New J. Phys. , 11, (3), 033027. Google Scholar CrossRef Search ADS   40. Pusch, A., Weber, S. & Porto, M. (2008) Impact of topology on the dynamical organization of cooperation in the prisoner’s dilemma game. Phys. Rev. E , 77, (3), 036120. Google Scholar CrossRef Search ADS   41. Perc, M., Szolnoki, A. & Szabó, G. (2008) Restricted connections among distinguished players support cooperation. Phys. Rev. E , 78, (6), 066101. Google Scholar CrossRef Search ADS   42. Roca, C. P., Cuesta, J. A. & Sánchez, A. (2009) Promotion of cooperation on networks? The myopic best response case. Eur. Phys. J. B , 71, (4), 587– 595. Google Scholar CrossRef Search ADS   43. Kim, B. J., Trusina, A., Holme, P., Minnhagen, P., Chung, J. S. & Choi, M. (2002) Dynamic instabilities induced by asymmetric influence: prisoners’ dilemma game in small-world networks. Phys. Rev. E , 66, (2), 021907. Google Scholar CrossRef Search ADS   44. Santos, F. C., Rodrigues, J. & Pacheco, J. M. (2005) Epidemic spreading and cooperation dynamics on homogeneous small-world networks. Phys. Rev. E , 72, (5), 056128. Google Scholar CrossRef Search ADS   45. Szolnoki, A., Perc, M., Szabó, G. & Stark, H.-U. (2009) Impact of aging on the evolution of cooperation in the spatial prisoner’s dilemma game. Phys. Rev. E , 80, (2), 021901. Google Scholar CrossRef Search ADS   46. Fu, F., Chen, X., Liu, L. & Wang, L. (2007) Social dilemmas in an online social network: the structure and evolution of cooperation. Phys. Lett. A , 371, (1), 58– 64. Google Scholar CrossRef Search ADS   47. Holme, P., Trusina, A., Kim, B. J. & Minnhagen, P. (2003) Prisoners’ dilemma in real-world acquaintance networks: Spikes and quasiequilibria induced by the interplay between structure and dynamics. Phys. Rev. E , 68, (3), 030901. Google Scholar CrossRef Search ADS   48. Lozano, S., Arenas, A. & Sánchez, A. (2008) Mesoscopic structure conditions the emergence of cooperation on social networks. PLoS ONE , 3, (4), e1892. Google Scholar CrossRef Search ADS   49. Wang, Z., Wang, L., Szolnoki, A. & Perc, M. (2015) Evolutionary games on multilayer networks: a colloquium. Eur. Phys. J. B , 88, (5), 1– 15. 50. Hamilton, W. (1964) The genetical evolution of social behaviour. I. J. Theoret. Biol. , 7, 1– 16. Google Scholar CrossRef Search ADS   51. Dugatkin, L. A. & Mesterton-Gibbons, M. (1996) Cooperation among unrelated individuals: reciprocal altruism, by-product mutualism and group selection in fishes. BioSystems , 37, (1), 19– 30. Google Scholar CrossRef Search ADS   52. Traulsen, A., Röhl, T. & Schuster, H. G. (2004) Stochastic gain in population dynamics. Phys. Rev. Lett. , 93, (2), 028701. Google Scholar CrossRef Search ADS   53. Pacheco, J. M., Traulsen, A., Ohtsuki, H. & Nowak, M. A. (2008) Repeated games and direct reciprocity under active linking. J. Theoret. Biol. , 250, (4), 723– 731. Google Scholar CrossRef Search ADS   54. Nowak, M. A. & Sigmund, K. (1998) The dynamics of indirect reciprocity. J. Theoret. Biol. , 194, (4), 561– 574. Google Scholar CrossRef Search ADS   55. Nowak, M. A. & Sigmund, K. (2005) Evolution of indirect reciprocity. Nature , 437, (7063), 1291– 1298. Google Scholar CrossRef Search ADS   56. Perc, M. & Szolnoki, A. (2008) Social diversity and promotion of cooperation in the spatial prisoner’s dilemma game. Phys. Rev. E , 77, (1), 011904. Google Scholar CrossRef Search ADS   57. Santos, F. C., Santos, M. D. & Pacheco, J. M. (2008) Social diversity promotes the emergence of cooperation in public goods games. Nature , 454, (7201), 213– 216. Google Scholar CrossRef Search ADS   58. Yang, H.-X., Wang, W.-X., Wu, Z.-X., Lai, Y.-C. & Wang, B.-H. (2009) Diversity-optimized cooperation on complex networks. Phys. Rev. E , 79, (5), 056107. Google Scholar CrossRef Search ADS   59. Chen, Y., Qin, S.-M., Yu, L., Zhang, S. (2008) Emergence of synchronization induced by the interplay between two prisoner’s dilemma games with volunteering in small-world networks. Phys. Rev. E , 77, (3), 032103. Google Scholar CrossRef Search ADS   60. Hauert, C., De Monte, S., Hofbauer, J. & Sigmund, K. (2002) Volunteering as red queen mechanism for cooperation in public goods games. Science , 296, (5570), 1129– 1132. Google Scholar CrossRef Search ADS   61. Hauert, C., Traulsen, A., Brandt, H., Nowak, M. A. & Sigmund, K. (2007) Via freedom to coercion: the emergence of costly punishment. Science , 316, (5833), 1905– 1907. Google Scholar CrossRef Search ADS   62. Szabó, G. & Tőke, C. (1998) Evolutionary prisoner’s dilemma game on a square lattice. Phys. Rev. E , 58, (1), 69. Google Scholar CrossRef Search ADS   63. Fudenberg, D., Nowak, M. A., Taylor, C. & Imhof, L. A. (2006) Evolutionary game dynamics in finite populations with strong selection and weak mutation. Theoret. Popul. Biol. , 70, (3), 352– 363. Google Scholar CrossRef Search ADS   64. Traulsen, A., Nowak, M. A. & Pacheco, J. M. (2007) Stochastic payoff evaluation increases the temperature of selection. J. Theoret. Biol. , 244, (2), 349– 356. Google Scholar CrossRef Search ADS   65. Bastian, M., Heymann, S., Jacomy, M. (2009) Gephi: an open source software for exploring and manipulating networks.. ICWSM , 8, 361– 362. 66. Ren, J., Wang, W.-X. & Qi, F. (2007) Randomness enhances cooperation: a resonance-type phenomenon in evolutionary games. Phys. Rev. E , 75, (4), 045101. Google Scholar CrossRef Search ADS   67. Vukov, J., Szabó, G. & Szolnoki, A. (2006) Cooperation in the noisy case: Prisoner’s dilemma game on two types of regular random graphs. Phys. Rev. E , 73, (6), 067103. Google Scholar CrossRef Search ADS   68. Khoo, T., Fu, F. & Pauls, S. (2016) Coevolution of cooperation and partner rewiring range in spatial social networks. Sci. Rep. , 6, 36293. Google Scholar CrossRef Search ADS PubMed  69. Gleeson, J. P. (2013) Binary-state dynamics on complex networks: pair approximation and beyond. Phys. Rev. X , 3, (2), 021004. 70. Zhou, J., Xiao, G. & Chen, G. (2013) Link-based formalism for time evolution of adaptive networks. Phys. Rev. E , 88, (3), 032808. Google Scholar CrossRef Search ADS   © The authors 2017. Published by Oxford University Press. All rights reserved. http://www.deepdyve.com/assets/images/DeepDyve-Logo-lg.png Journal of Complex Networks Oxford University Press

# Evolutionary prisoner’s dilemma games coevolving on adaptive networks

, Volume 6 (1) – Feb 1, 2018
23 pages

Publisher
Oxford University Press
ISSN
2051-1310
eISSN
2051-1329
D.O.I.
10.1093/comnet/cnx018
Publisher site
See Article on Publisher Site

### Abstract

Abstract We study a model for switching strategies in the prisoner’s dilemma game on adaptive networks of player pairings that coevolve as players attempt to maximize their return. We use a node-based strategy model wherein each player follows one strategy at a time (cooperate or defect) across all of its neighbours, changing that strategy and possibly changing partners in response to local changes in the network of player pairing and in the strategies used by connected partners. We compare and contrast numerical simulations with existing pair approximation differential equations for describing this system, as well as more accurate equations developed here using the framework of approximate master equations. We explore the parameter space of the model, demonstrating the relatively high accuracy of the approximate master equations for describing the system observations made from simulations. We study two variations of this partner-switching model to investigate the system evolution, predict stationary states and compare the total utilities and other qualitative differences between these two model variants. 1. Introduction Game theory is the study of strategic decision making and the analysis of mathematical models of conflict and cooperation between intelligent rational participants [1–3]. Game theory is used in a variety of disciplines, including biology, economics, political science, computer science and psychology. In classical game theory, rational actors make their choices to optimize their individual payoffs. That is, actors make strategic choices on a rationally determined evaluation of probable outcomes, or utility, considering the strategic analysis that the players’ opponents are making in determining their own choices. Evolutionary game theory [4–8], the application of game theory to evolving populations, has recently expanded from consideration of life forms in biology to various areas in social science. Evolutionary game theory is useful in this context by defining a framework of contests, strategies, behaviours and analytics in which competition can be modelled. The key point in evolutionary game theory is that the success of a strategy is not just determined by how good the strategy is in itself; rather, it is a question of how good the strategy is in the presence of other alternative strategies and of the distribution of those other strategies within a population. One of the great difficulties of Darwinian theory, recognized by Darwin himself, was the problem of altruism [9]; in particular, exploited cooperators are worse off than defectors. Hence, according to the basic principles of Darwinian selection, intuitively it seems almost certain that cooperation should go extinct. If the basis for selection is at the individual level, the phenomenon of altruism is often hard to interpret. And selection at the group level, or for the ‘greater good’, appears to violate the game theory assumption of individuals maximizing their own utility. Indeed, altruism is certainly not found to be the general case in nature. Nevertheless, altruistic behaviours can be found in many social animals and, indeed, can be fundamental for some species to survive [10]. The solution to this apparent paradox can be demonstrated in the application of evolutionary game theory to the prisoner’s dilemma game [11, 12], a game which tests the outcomes of ‘cooperating’ versus ‘defecting’. Cooperation is usually analyzed in game theory by means of a non-zero-sum game. The prisoner’s dilemma game, one of the most studied systems in all of game theory, is a standard example that shows why two completely rational individuals may not cooperate, even if it appears that it is in their best collective interest to do so. Within evolutionary game theory, the analysis of the prisoner’s dilemma is as an iterative game [13], with the repetitive nature affording competitors the possibility of retaliating or defecting based on the results of previous rounds of the game. There are a multitude of strategies which have been tested by the mathematics of evolutionary game theory and in computer simulations of contests (see, e.g. [14]), with the general conclusion that the most effective competitive strategies are typically cooperative with a reserved retaliatory response as necessary. The most famous and one of the most successful of these strategies is ‘Tit for Tat’, which carries out this approach by executing a simple algorithm [11, 15]. In recent years, games played on various random graphs and social networks have been investigated [16–25, 27, 28]. For example, the public goods game can be seen as a generalization of multiplayer N-player games and group interactions such as pattern formation and self-organization could be studied through this extension [29–31]. The studies devoted to evolutionary games on complex networks are extensive. Speaking generally, the goals of such studies typically include identifying which combination of game rules, dynamics and various network topologies can provide cooperation among selfish and unrelated individuals. In the present work, we focus on the dynamical role of players having the option to switch partners, as studied in the framework of adaptive networks that coevolve with the player strategies [32–37]. On a ‘coevolving’ or ‘adaptive’ network with a game, the vertices represent players and the edges denote the pairings, or game interactions, between players. Importantly, the spatial structures in a network could enable cooperators to form small groups or clusters to protect themselves against exploitation by defectors [25]. Studies have shown that the spatial structure can promote cooperation [26], but there is also evidence that suggests that spatial structure may not necessarily favour cooperation [20]. Other studies have elaborated on different aspects of cooperation on scale-free [38–40], square-lattice [41, 42], small-world [43–45], social and real-world networks [46–48] and multilayer networks [49]. Insofar as the player pairing network is part of a larger social setting, other notable rules encouraging cooperative behaviour are kin selection [50], group selection [51, 52], direct reciprocity [53], indirect reciprocity [54, 55], social diversity [56–58], voluntary participation [59–61], reputation [18] and frequency of breaking cooperator-defector bonds and rewiring [36, 37]. All of these have been studied as interesting mechanisms that may promote cooperation in evolutionary games. In this article, we focus our study on a simple coevolving network model for how players play a prisoner’s dilemma game with the ability to adapt by changing their strategies or switching partners if they are exploited by their neighbours, introduced in [19]. We are interested in exploring the cooperative level among the individuals as they organize into networks of cooperators. By using the technique of approximate master equations, we provide a more accurate approximation of the evolution of the network structure and player states with which to explore the parameter space of the model. We compare the existing analytical methods and our new approximation, providing qualitative and quantitative estimations of various network properties. In Section 2, we introduce the model. We then describe our more accurate approximation in Section 3. In Section 4, we provide numerical results and compare the two approximation methods. In Section 5, we introduce a variation to the model and compare the results of the two variants of the model. Finally, we make concluding remarks in Section 6. 2. Description of the Cooperative-Defective (CD)-switching model We study the partner switching model introduced by Fu et al. [19], where vertices of the coevolving network represent players, edges denote the pairing of individuals playing the game, and players adapting by changing strategies based on their current local information or switching partners. For simplicity, we initialize our simulations with an Erdős–Rényi network with $$N$$ nodes and $$M$$ edges, with each node assigned an initial state either as a cooperator ($$C$$) or defector ($$D$$). Each node uses a single strategy across all of its links, though that strategy can change over time as we describe below. That is, individual $$i$$ playing with all of her connections obtains an income   Pi=∑j∈NisiTPsj, where $$\mathscr{N}_{i}$$ is the neighbourhood set of $$i$$ and the 2-by-2 payoff matrix $$P$$ is   CD(C11+uD0u) with cost-benefit ratio $$u \in (0,1)$$ determining the relative outcomes in the payoff matrix. That is, if players $$i$$ and $$j$$ in a pairing both play $$C$$, then both receive the payoff $$1$$ from this pairing; if player $$i$$ plays $$C$$ and player $$j$$ plays $$D$$, then player $$i$$ gets payoff $$0$$ and player $$j$$ gets payoff $$1+u$$; and so on. Each time step of our microsimulation proceeds as follows. We uniformly at random pick one of the edges that connects a pair of players with different strategies, that is a $$CD$$ link, denoted by $$E_{ij}$$. (In Section 5, we study a variant of the model where both $$CD$$ and $$DD$$ links are considered.) With probability $$w$$ (specified as a parameter), the two nodes $$i$$ and $$j$$ connected by edge $$E_{ij}$$ consider updating their strategies; otherwise (i.e. with probability $$1-w$$), the edge $$E_{ij}$$ is rewired. When a node reassesses its strategy along the edge $$E_{ij}$$, the node has probability $$\phi$$ specified by the Fermi function to change its state, as first proposed by [62], as specified in detail below. When link $$E_{ij}$$ is rewired, the player with end state $$C$$ unilaterally drops the partnership with its neighbour with end state $$D$$ on the edge $$E_{ij}$$ and picks (uniformly at random) another player from the remaining population outside its immediate neighbourhood as its new partner. The left panel of Fig. 1 illustrates this rewiring process. Fig. 1. View largeDownload slide (Color online) Illustrations of the rewiring and strategy updating processes. Pick a discordant edge $$XY$$ with node $$X$$ in state $$C$$ and node $$Y$$ in state $$D$$. (Left) Rewiring occurs with probability $$1-w$$ with cooperating node $$X$$ dismissing its defecting neighbour $$Y$$ and rewiring to a random node $$Z$$ in the network, independent of the state of node $$Z$$. If $$Z$$ is in state $$D$$, then the number of $$CD$$ edges remains the same; but if $$Z$$ is cooperating (as in the picture), then the number of $$CD$$ edges has been reduced. (Right) Strategy updating occurs with probability $$w$$. Nodes $$X$$ and $$Y$$ compare their utilities, with one node copying the other as selected according to the Fermi function of the difference of utilities. In the example depicted here, suppose the cost-benefit ratio $$u = 0.5$$ and the Fermi function bias parameter $$\alpha=30$$. Then node $$X$$ has utility $$P_X=1$$, node $$Y$$ has utility $$P_Y=4$$, and the Fermi function gives $$\phi(s_{X} \leftarrow s_{Y}) = {1}/({1+\exp[\alpha(P_{X}-P_{Y})]}) = {1}/({1+\exp[30(1-4)]}) \approx 1$$, so that node $$X$$ is selected with probability close to 1 to imitate node $$Y$$’s strategy (state $$D$$). Fig. 1. View largeDownload slide (Color online) Illustrations of the rewiring and strategy updating processes. Pick a discordant edge $$XY$$ with node $$X$$ in state $$C$$ and node $$Y$$ in state $$D$$. (Left) Rewiring occurs with probability $$1-w$$ with cooperating node $$X$$ dismissing its defecting neighbour $$Y$$ and rewiring to a random node $$Z$$ in the network, independent of the state of node $$Z$$. If $$Z$$ is in state $$D$$, then the number of $$CD$$ edges remains the same; but if $$Z$$ is cooperating (as in the picture), then the number of $$CD$$ edges has been reduced. (Right) Strategy updating occurs with probability $$w$$. Nodes $$X$$ and $$Y$$ compare their utilities, with one node copying the other as selected according to the Fermi function of the difference of utilities. In the example depicted here, suppose the cost-benefit ratio $$u = 0.5$$ and the Fermi function bias parameter $$\alpha=30$$. Then node $$X$$ has utility $$P_X=1$$, node $$Y$$ has utility $$P_Y=4$$, and the Fermi function gives $$\phi(s_{X} \leftarrow s_{Y}) = {1}/({1+\exp[\alpha(P_{X}-P_{Y})]}) = {1}/({1+\exp[30(1-4)]}) \approx 1$$, so that node $$X$$ is selected with probability close to 1 to imitate node $$Y$$’s strategy (state $$D$$). When a strategy updating event occurs, nodes $$i$$ and $$j$$ each consider their play across all of their neighbours, observing their total payoffs $$P_{i}$$ and $$P_{j}$$. Then the strategy of node $$j$$ replaces that of $$i$$ along the edge $$E_{ij}$$—that is, $$i$$ copies $$j$$’s strategy—with probability given by the Fermi function   ϕ(si←sj)=11+exp⁡[α(Pi−Pj)], where $$\alpha$$ modifies the intensity of the bias towards the strategy with the higher payoff [63, 64]. Otherwise [that is, with probability $$\phi(s_{j} \leftarrow s_{i}) = 1 - \phi(s_{i} \leftarrow s_{j})$$], $$j$$ copies $$i$$. The value of $$1/\alpha$$ can be interpreted here as representing the amplitude of noise in the strategy updating process [66, 67]; that is, $$\alpha \rightarrow 0$$ ignores the payoffs while $$\alpha \rightarrow \infty$$ yields deterministic imitation of the node receiving the higher payoff. The right panel of Fig. 1 illustrates the strategy updating process. This partner switching evolutionary game stops when there are no discordant edges remaining in the network. That is, only $$CC$$ and $$DD$$ edges exist in the final state of the system, with the final network fissioning these two groups (cooperative and defective) into different components. In Fig. 2, we visualize a simulated network shortly before fission occurs, showing strong grouping of nodes by strategies. Fig. 2. View largeDownload slide (Color online) Visualization of the CD-switching coevolving network model shortly before the network fissions into two disconnected components, one including all of the remaining cooperator nodes (light/blue) and the other containing only defecting nodes (dark/red). This simulation contains $$N \,{=}\, 1,000$$ nodes and $$M \,{=}\, 5,000$$ edges, with cost-benefit ratio $$u \,{=}\, 0.5$$, strategy updating probability $$w \,{=}\, 0.1$$, and initial fraction of defectors $$\rho \,{=}\, 0.5$$. This visualization was created using the Yifan Hu layout in Gephi [65]. Fig. 2. View largeDownload slide (Color online) Visualization of the CD-switching coevolving network model shortly before the network fissions into two disconnected components, one including all of the remaining cooperator nodes (light/blue) and the other containing only defecting nodes (dark/red). This simulation contains $$N \,{=}\, 1,000$$ nodes and $$M \,{=}\, 5,000$$ edges, with cost-benefit ratio $$u \,{=}\, 0.5$$, strategy updating probability $$w \,{=}\, 0.1$$, and initial fraction of defectors $$\rho \,{=}\, 0.5$$. This visualization was created using the Yifan Hu layout in Gephi [65]. We note that recent work [68] studied a spatially embedded extension of the partner switching model in [19], introducing a range of rewiring distance and finding that a preference for global partner switching can coevolve with cooperation. However, in the present contribution we consider only random rewiring partner switching. 3. Semi-analytical methods of approximation We study this generalized coevolving network model with simulations and approximate model equations. The frameworks of mean field theory, pair approximation equations (PA) and approximate master equations (AME) have all been used effectively in similar settings. The PA equations for these dynamics were obtained by [19]. Among these three levels of approximation, AME can often be used to achieve the greatest accuracy [69], motivating us to develop the AME system here. For comparison with the AME system that we will specify, we first look at the PA equations derived in [19], describing the dynamics of the quantities $$N_{X}$$ and $$N_{XY}$$—where $$X,Y \in \{ C, D\}$$—that count the numbers of nodes and edges corresponding to the two node states. For example, $$N_{C}$$ denotes the number of cooperators and $$N_{CC}$$ denotes the number of $$CC$$ links in the network. While the node states and network topology coevolve, the total numbers of nodes ($$N$$) and edges ($$M$$) remain conserved by the specified dynamics, requiring $$N_{C} + N_{D} = N$$ and $$N_{CC} + N_{CD} + N_{DD} = M$$. The evolution of these counts is then obtained through a moment closure approximating triple counts $$N_{XYZ}$$—where $$X, Y, Z \in \{ C, D\}$$—in terms of the edge counts $$N_{CC}$$, $$N_{CD}$$ and $$N_{DD}$$. Specifically, $$N_{XXY}$$ is approximated by the product of $$XX$$ links and the average number of $$XY$$ links that an $$X$$ node has, that is $$N_{XY}/N_{X}$$, hence $$N_{XXY} = 2N_{XX}N_{XY}/N_{X}$$. The resulting PA equations [19] are as follows:   dNCdt =w⋅NCD⋅tanh⁡[α2(π¯C−π¯D)]dNCCdt =w⋅(NCDϕC→D−2NCDNCCNCϕD→C+NCDNCDNDϕC→D) +(1−w)⋅NCNNCDdNDDdt =w⋅(NCDϕD→C−2NCDNDDNDϕC→D+NCDNCDNCϕD→C), (3.1) where $$w$$ is the probability of strategy updating (versus edge rewiring), $$\overline{\pi}_{C} = 1 \cdot {2N_{CC}}/{N_{C}} + 0 \cdot {N_{CD}}/{N_{D}}$$ is the average utility of $$C$$ nodes, $$\overline{\pi}_{D} = (1+u) \cdot {N_{CD}}/{N_{D}} + u \cdot {2N_{DD}}/{N_{D}}$$ is the average utility of $$D$$ nodes, $$\phi_{C\to D} = 1/(1+\exp [\alpha (\overline{\pi}_{D} - \overline{\pi}_{C})])$$, $$\phi_{D\to C} = 1/(1+\exp [\alpha (\overline{\pi}_{C} - \overline{\pi}_{D})])$$, and $$\alpha$$ controls the intensity of selection. The AME method has been successfully used to approximate various processes on static and coevolving networks. The difficulty in deriving the AME system for the dynamics studied here is that the present transition probabilities depend on more than nearest neighbour states. For comparison, in the Susceptible-Infected-Susceptible (SIS) model the recovery rate of an infected (I) node and the probability of infection along a given SI edge are both specified as constant parameters. Similarly, in typical voter model dynamics, the probability for a node to change its state relies on its neighbours’ opinions. In contrast, in the partner switching evolutionary game studied here, the local information impacting the decision of a node to change its state includes its total utility and that of its neighbours, which depend on the states of the neighbours’ neighbours. Since the binary-state AME framework only includes the node states, degrees and neighbour states, we need estimate neighbours’ utilities in terms of these limited quantities. We will explain how we approximate these utilities after we introduce the equations below. Let $$C_{k,l}(t)$$ and $$D_{k,l}(t)$$, respectively, be the number of cooperating and defecting nodes of degree $$k$$ with $$l$$ defecting neighbours at time $$t$$. We note that the numbers of nodes using each strategy are then given by the zeroth moments of the $$C_{k,l}(t)$$ and $$D_{k,l}(t)$$ distributions, $$N_{C} = \sum_{kl}C_{k,l}$$ and $$N_{D} = \sum_{kl}D_{k,l}$$. Meanwhile, the first moments give the numbers of edges of each type: $$N_{CC} = \frac{1}{2} \sum_{kl}(k-l)C_{k,l}$$, $$N_{CD} = \sum_{kl}lC_{k,l} = \sum_{kl}(k-l)D_{k,l}$$ and $$N_{DD} = \frac{1}{2} \sum_{kl}lD_{k,l}$$. Importantly, while the node states and network topology coevolve, the total numbers of nodes and edges remain conserved by the specified dynamics, requiring $$N_{C} + N_{D} = N$$ and $$N_{CC} + N_{CD} + N_{DD} = M$$. The AME system of ordinary differential equations governing the $$C_{k,l}(t)$$ and $$D_{k,l}(t)$$ compartments is of course more complicated than the (relatively) compact PA equations (3.1). In the following paragraphs, we describe each of the terms that appear in this AME system:   dCk,ldt =w{ϕk,lD⋅(k−l)Dk,l−ϕk,lC⋅lCk,l +ϕCD←CC⋅γC(l+1)Ck,l+1−ϕCD←CC⋅γClCk,l +ϕCC←CD⋅βC(k−l+1)Ck,l−1−ϕCC←CD⋅βC(k−l)Ck,l} +(1−w){NCN[(l+1)Ck,l+1−lCk,l]+NCDN[Ck−1,l−Ck,l]}, (3.2)  dDk,ldt =w{−ϕk,lD⋅(k−l)Dk,l+ϕk,lC⋅lCk,l +ϕDD←DC⋅γD(l+1)Dk,l+1−ϕDD←DC⋅γDlDk,l +ϕDC←DD⋅βD(k−l+1)Dk,l−1−ϕDC←DD⋅βD(k−l)Dk,l} +(1−w){[(k−l+1)Dk+1,l−(k−l)Dk,l]+NCDN[Dk−1,l−Dk,l]}. (3.3) To describe the AME derivation, we focus on explaining the $$C_{k,l}$$ equation, as the effects of corresponding terms of the $$D_{k,l}$$ equation are similar. The first three lines of equation (3.2), premultiplied by $$w$$, are the effect of strategy updating, while the last line with $$1-w$$ are the effect of rewiring. To write the AME system, we track all counts flowing in and out of ‘centre’ class, $$C_{k,l}$$. As visualized in Fig. 3, there are six such flows for the $$C_{k,l}$$ compartment, yielding the first six terms in the $$C_{k,l}$$ equation. The first term describes the rate at which $$D_{k,l}$$ nodes change to $$C_{k,l}$$ through comparing its utility with one of its $$C$$ neighbours, with the Fermi function evaluation $$\phi^{D}_{k,l}$$ as estimated below. Similarly, the second term captures $$C_{k,l}$$ nodes changing strategy to become $$D_{k,l}$$ after comparing their utility with their $$D$$ neighbours, with Fermi function evaluation $$\phi^{C}_{k,l}$$ also estimated below. Fig. 3. View largeDownload slide (Color online) Illustration of transitions to and from the $$C_{k,l}$$ and $$D_{k,l}$$ compartments (cooperating and defecting nodes, respectively, with degree $$k$$ and $$l$$ defecting neighbours) in the system of AME due to players updating their strategies. For each compartment, only a subset of neighbours in the corresponding ego network are depicted here, to signal the essential changes corresponding to each flow. The term describing each flow is indicated above/below the corresponding line connecting compartments. Fig. 3. View largeDownload slide (Color online) Illustration of transitions to and from the $$C_{k,l}$$ and $$D_{k,l}$$ compartments (cooperating and defecting nodes, respectively, with degree $$k$$ and $$l$$ defecting neighbours) in the system of AME due to players updating their strategies. For each compartment, only a subset of neighbours in the corresponding ego network are depicted here, to signal the essential changes corresponding to each flow. The term describing each flow is indicated above/below the corresponding line connecting compartments. To estimate these Fermi function transition probabilities, we can directly compute the utility of the centre node, since we know the numbers and types of its neighbours; but we need to estimate neighbours’ utilities in terms of the limited information available in the AME framework. Recall that the payoff matrix of our prisoner’s dilemma game is   CD(C11+uD0u), where $$u \in (0,1)$$. For transitions by a strategy update out of the $$D_{k,l}$$ class, the centre node is of state $$D$$, with $$k$$ neighbours, $$l$$ of which are of state $$D$$. Hence we can compute the centre’s utility as   PD=l⋅u+(k−l)⋅(1+u). Since we do not have any information about the neighbours of the $$C$$ neighbours of these $$D_{k,l}$$ nodes, we approximate utility by estimating the numbers and types of the neighbours of a $$C$$ neighbour. Specifically, we denote the expected number of $$C$$ neighbours of this $$C$$ neighbour by $$\eta^{D}$$, and the expected number of $$D$$ neighbours of a $$C$$ neighbour by $$\beta^{D} + 1$$, where $$\eta^{D}$$ and $$\beta^{D}$$ will be estimated below and the $$+1$$ accounts for the $$D_{k,l}$$ node itself. The estimate of the utility of the $$C$$ neighbour thus becomes   P^C=ηD⋅1+(βD+1)⋅0 and we compare the centre node $$D_{k,l}$$’s utility with the estimate from one of its $$C$$ neighbours via the Fermi function, so that the estimated probability of the $$D_{k,l}$$ node changing its state in this consideration is   ϕk,lD=11+exp⁡[α(PD−P^C)]=11+exp⁡[α(lu+(k−l)(1+u)−ηD)]. Meanwhile, transitions by a strategy update out of the $$C_{k,l}$$ rely on the utility of a $$C_{k,l}$$ node,   PC=l⋅0+(k−l)⋅1, and estimating the utility of one of the $$D$$ neighbours via the numbers and types of its neighbours. We denote the expected number of $$C$$ neighbours of this $$D$$ neighbour (including the $$C_{k,l}$$ node in question) by $$\gamma^{C}$$ + 1, and the number of $$D$$ neighbours of this $$D$$ neighbour as $$\delta^{C}$$, both of which will be estimated below. With this notation, the expected utility of one of the $$D$$ neighbours is   P^D=(γC+1)⋅(1+u)+δC⋅u and the Fermi function estimate for the probability of the $$C_{k,l}$$ node changing its state in this comparison becomes   ϕk,lC=11+exp⁡[α(PC−P^D)]=11+exp⁡[α(k−l−(γC+1)(1+u)+δCu)]. The third–sixth terms in the $$C_{k,l}$$ equation are the effects of one of the neighbours changing its state and thus leading to a change of the total quantity of $$C_{k,l}$$. Similar to the above, in order to make estimates of the transition probabilities we need a variety of estimates of the numbers and types of the neighbours’ neighbours, which we denote as follows. We use $$\beta^{C}$$ to provide the expected number of $$D$$ neighbours of a $$CC$$ edge, as   βC=∑k,l(k−l)lCk,l∑k,l(k−l)Ck,l. Similarly, $$\beta^{D}$$ denotes the expected number of $$D$$ neighbours of the $$C$$ node in a $$CD$$ edge by   βD=∑k,ll2Ck,l∑k,llCk,l. Meanwhile, $$\gamma^{C}$$ and $$\gamma^{D}$$ estimate the numbers of $$C$$ neighbours of the $$D$$ node of a $$CD$$ edge or of a $$DD$$ edge, respectively:   γC=∑k,l(k−l)2Dk,l∑k,l(k−l)Dk,l,γD=∑k,ll(k−l)Dk,l∑k,llDk,l. Similarly, $$\delta^{C}$$ and $$\delta^{D}$$ provide the expected numbers of $$D$$ neighbours of the $$D$$ node of a $$CD$$ edge or of a $$DD$$ edge, respectively, while $$\eta^{C}$$ and $$\eta^{D}$$ give the numbers of $$C$$ neighbours of a $$CC$$ edge or the $$C$$ node of a $$CD$$ edge, respectively:   δC=∑k,l(k−l)lDk,l∑k,l(k−l)Dk,l,δD=∑k,ll2Dk,l∑k,llDk,l,ηC=∑k,l(k−l)2Ck,l∑k,l(k−l)Ck,l,ηD=∑k,ll(k−l)Ck,l∑k,llCk,l. To compute the corresponding $$\phi$$ transition probability in each term, let us first consider the Fermi function factors on the second line in equation (3.2) specified by   ϕCD←CC=11+exp⁡[α(PDCD←CC−πC)], where   PDCD←CC=(γC+1)⋅(1+u)+δC⋅u and   πC=1⋅2NCCNC+0⋅NCDNC. That is, this transition probability considers a $$D$$ neighbour of the centre $$C_{k,l}$$ class, and we suppose this $$D$$ neighbour has $$\gamma^{C}+1$$ neighbours in state $$C$$ (including the centre $$C_{k,l}$$ node) and $$\delta^{C}$$ in state $$D$$. Then we estimate this $$D$$ node’s utility and compare it with its typical $$C$$ neighbour (that is, the average utility $$\pi_{C}$$). We note for emphasis here that other choices could have been made in this estimate; specifically, we explicitly know that one of the $$C$$ neighbours of this $$D$$ node is the centre $$C_{k,l}$$ node. An alternative formulation of the AME system could be formed that separates out the state change of the $$D$$ neighbour in comparison with the $$C_{k,l}$$ centre (thus removing the +1 contribution) from its state changes through interaction with other $$C$$ nodes. In order to keep the equations just a little simpler, and because of the good approximation we observe in our results below, we continue to employ the simpler estimate here. We similarly estimate the Fermi function factors on the third line in equation (3.2) as   ϕCC←CD=11+exp⁡[α(PCCC←CD−πD)], where   PCCC←CD=(ηC+1)⋅1+βC⋅0 and   πD=(1+u)⋅NCDND+u⋅2NDDND. In analogy to the above argument, we consider a $$C$$ neighbour of the centre $$C_{k,l}$$ class and suppose this $$C$$ node has $$\eta^{C}+1$$ neighbours in state $$C$$ (including the centre node) and $$\beta^{C}$$ in state $$D$$. We estimate this $$C$$ node’s utility and compare it with the average utility of a $$D$$ node, $$\pi_{D}$$. The last line in the $$C_{k,l}$$ equation (3.2) describes the rewiring effect. The class $$C_{k,l+1}$$ flows to $$C_{k,l}$$ when the centre node $$C$$ drops one of its defecting neighbours and rewires randomly to another $$C$$. Similarly, the $$C_{k,l}$$ count decreases when the centre node $$C$$ drops one of its defecting neighbours and rewires randomly to another $$C$$. If the rewiring instead links to another $$D$$ node, these counts do not change. Finally, a $$C_{k-1,l}$$ node becomes $$C_{k,l}$$ if it is the recipient of a rewired edge (from another $$C$$ node), while the $$C_{k,l}$$ count decreases if a $$C_{k,l}$$ receives a rewired edge. The first group of square-bracketed terms on this line corresponds to the centre node actively rewiring to some other node in the network while the second group in square brackets describes the centre node passively receiving a rewired edge through the action of some other node in the network. Figure 4 illustrates these active and passive rewiring processes. Fig. 4. View largeDownload slide (Color online) Illustrations of nodes undergoing active and passive rewiring. (Left) Active rewiring: before the rewiring, node $$X$$ is in class $$C_{k,l}$$ (cooperator of degree $$k$$ with $$l$$ defecting neighbours). Suppose one of $$X$$’s discordant edges is selected and X actively breaks that edge shared with a defective neighbour, rewiring to a random node in the network. If node $$X$$ rewires to a node of state $$C$$ (as depicted here), then node $$X$$ moves to class $$C_{k,l-1}$$. We recall that only $$C$$ nodes actively rewire in the present model. (Right) Passive rewiring: before the rewiring, node $$X$$ is in class $$C_{k,l}$$. Suppose in the rewiring process, node $$X$$ is passively rewired through the activity of another $$C$$ node in the network. Then node $$X$$ moves to class $$C_{k+1,l}$$. Fig. 4. View largeDownload slide (Color online) Illustrations of nodes undergoing active and passive rewiring. (Left) Active rewiring: before the rewiring, node $$X$$ is in class $$C_{k,l}$$ (cooperator of degree $$k$$ with $$l$$ defecting neighbours). Suppose one of $$X$$’s discordant edges is selected and X actively breaks that edge shared with a defective neighbour, rewiring to a random node in the network. If node $$X$$ rewires to a node of state $$C$$ (as depicted here), then node $$X$$ moves to class $$C_{k,l-1}$$. We recall that only $$C$$ nodes actively rewire in the present model. (Right) Passive rewiring: before the rewiring, node $$X$$ is in class $$C_{k,l}$$. Suppose in the rewiring process, node $$X$$ is passively rewired through the activity of another $$C$$ node in the network. Then node $$X$$ moves to class $$C_{k+1,l}$$. Shifting our focus to the six flows in and out of the set depicted in the centre of the lower row of Fig. 3, we similarly obtain equation (3.3) for the $$D_{k,l}$$ compartment. The resulting system contains $$2(k_\mathrm{max} + 1)^2$$ coupled differential equations, where $$k_\mathrm{max}$$ is the maximum degree a node can have in the network. Usually, the necessary value of $$k_\mathrm{max}$$ for good accuracy depends on network structure and the mean degree. Here we choose $$k_\mathrm{max} = 50$$. We numerically solve these differential equations to semi-analytically approximate the evolution of the system, using the ode45 solver in MATLAB until the solutions reach steady state. While we consider PA and AME approaches here, we note that [70] used an approach for studying adaptive SIS networks that treated the links as the objects, classifying them according to the disease states while tracking the degree and number of infected neighbours at both end nodes. This link-based method could generally improve the accuracy compared to AME; however, it also increases the number of coupled equations from $$O(k^{2}_\mathrm{max})$$ to $$O(k^{4}_\mathrm{max})$$, where $$k_\mathrm{max}$$ is the maximum degree in the network. 4. Simulations of the CD-switching model To test the accuracy of the approximations, we study simulated dynamics on networks with $$N = 1,000$$ nodes and $$M = 5,000$$ edges. That is, the mean degree of the network is fixed to be $$\langle k \rangle = 2M/N = 10$$. We set the parameter $$\alpha=30$$ in the Fermi function used for imitating strategies. We study dynamics for different initial fractions of defectors, denoted by $$\rho$$. We consider $$\rho = 0.5$$ first, and explore the influence of this parameter later. We draw the initial networks from an Erdős–Rényi $$G(N, M)$$ random graph model of $$N$$ nodes and $$M$$ edges distributed uniformly and independently between the nodes. For large $$N$$, the initial degree distribution is approximated by the Poisson distribution of mean $$\langle k \rangle$$, with the probability of a selected node having degree $$k$$ being $$p_{k} = {\langle k \rangle^{k}e^{-\langle k \rangle}}/{k!}.$$ In the present model, updating only discordant edges (i.e. $$CD$$), the dynamics stop when there are no discordant edges remaining. Unless stated otherwise, we perform $$1,000$$ simulations for each condition and report the average. 4.1. Final level of cooperation To study the effect of the cost-benefit ratio $$u$$ and strategy update rate $$w$$, we focus on the level of cooperation obtained in the final (stationary) states of the model, starting from an initial fraction of defectors $$\rho=0.5$$. In Fig. 5, we present heat maps generated by simulations and AME approximations of the final fractions of cooperating nodes and CC links. As qualitatively expected, the final fraction of cooperating nodes is higher when the possible additional payoff for defecting, $$u$$, is smaller. When $$w$$ is close to $$0$$, we observe a moderate level of cooperators, as expected because the discordant edges can resolve themselves by rewiring with fewer strategy updates. Away from the small values of $$u$$ or $$w$$, the final network states are dominated by defectors. The AME approximation generally captures these high- and low-cooperation regions of the $$(u,w)$$ parameter space, up to a modest displacement in the precise position of the phase transition. Fig. 5. View largeDownload slide (Color online) Results from simulations (left column) and approximate master equations (AME) (right column) of the final fractions of cooperators (top row) and CC edges (bottom row) for different combinations of the cost-benefit ratio $$u$$ and strategy updating probability $$w$$. The initial fraction of defectors is $$\rho=0.5$$. For both the $$u$$ and $$w$$ axes, we use steps of 0.05 and plot the results from stationary states. Simulation results here are averaged over 50 realizations at each parameter set. These visualizations were generated from results on a regular grid through bilinear interpolation, leading to some clearly apparent grid artefacts. While some discrepancies between simulation and AME results are clearly present, we note in particular that the position of the phase transition in the $$(u,w)$$ parameter space is well approximated by the AME system. Fig. 5. View largeDownload slide (Color online) Results from simulations (left column) and approximate master equations (AME) (right column) of the final fractions of cooperators (top row) and CC edges (bottom row) for different combinations of the cost-benefit ratio $$u$$ and strategy updating probability $$w$$. The initial fraction of defectors is $$\rho=0.5$$. For both the $$u$$ and $$w$$ axes, we use steps of 0.05 and plot the results from stationary states. Simulation results here are averaged over 50 realizations at each parameter set. These visualizations were generated from results on a regular grid through bilinear interpolation, leading to some clearly apparent grid artefacts. While some discrepancies between simulation and AME results are clearly present, we note in particular that the position of the phase transition in the $$(u,w)$$ parameter space is well approximated by the AME system. In Fig. 6, we further explore the final level of cooperation and assess the accuracies of the two semi-analytical approximations. In the left panel of Fig. 6, we consider $$w = 0$$, $$w = 0.05$$, $$w = 0.1$$ and $$w = 0.5$$, plotting the final fraction of nodes in state $$C$$ versus the cost-benefit ratio, $$u$$. There are three sets of outcomes here: simulations (markers), PA (dotted lines) and AME (dashed lines). For $$w = 0$$, there are no strategy updates and the final fraction of $$C$$ nodes is fixed at $$1-\rho$$. For $$w > 0$$, strategy updates come into play with larger values of the cost-benefit ratio $$u$$ driving stronger incentive for nodes to defect, decreasing the final fraction of cooperators. As seen in the figure, the cases $$w = 0.1$$ and $$w = 0.5$$ include higher levels of cooperation at small $$u$$ that decreases slightly as $$u$$ increases before suddenly dropping to zero near $$u \doteq 0.2$$ and $$u \doteq 0.6$$, respectively. This phenomenon is qualitatively captured by both PA and AME, though the quantitative description from AME is much more accurate, particularly in terms of these critical points. In contrast, the PA prediction of the fraction of cooperators drops to zero at much smaller values of $$u$$. Similarly, in the case $$w = 0.05$$, the level of cooperation in the simulation results decreases with increasing $$u$$, but without the sharp drop-off; in contrast to both the simulations and AME, the PA result drops to zero around $$u \doteq 0.9$$. Fig. 6. View largeDownload slide (Color online) (Left) Final fraction of cooperators versus cost-benefit ratio $$u$$ for different strategy updating probabilities $$w$$. Markers are averages from 1,000 simulations, dotted lines are pair approximation (PA) results, and dashed lines are from approximate master equations (AME). The PA results are different from those shown in [19], but we have confirmed the accuracy of our results by personal communication with those authors. (Right) Final fraction of cooperators versus $$w$$ for different cost-benefit ratio $$u$$. Markers, dotted lines, and dashed lines again indicate simulations, PA, and AME, respectively. In both panels, we observe that AME is typically more accurate than PA, especially so near the phase transition where the fraction of cooperators goes to zero. Fig. 6. View largeDownload slide (Color online) (Left) Final fraction of cooperators versus cost-benefit ratio $$u$$ for different strategy updating probabilities $$w$$. Markers are averages from 1,000 simulations, dotted lines are pair approximation (PA) results, and dashed lines are from approximate master equations (AME). The PA results are different from those shown in [19], but we have confirmed the accuracy of our results by personal communication with those authors. (Right) Final fraction of cooperators versus $$w$$ for different cost-benefit ratio $$u$$. Markers, dotted lines, and dashed lines again indicate simulations, PA, and AME, respectively. In both panels, we observe that AME is typically more accurate than PA, especially so near the phase transition where the fraction of cooperators goes to zero. Similarly, in the right panel of Fig. 6 we plot the final fraction of cooperators $$C$$ versus $$w$$ for different cost-benefit ratios, $$u = 0$$, $$u = 0.01$$, $$u = 0.2$$ and $$u = 0.8$$. For $$w=0$$, no nodes update their strategies and the final fraction of $$C$$ nodes is $$1-\rho=0.5$$. Larger $$w$$ increases both the number of strategy updates and their overall effect on the final state. In the cases $$u = 0$$ and $$u = 0.01$$, $$u$$ is so small that there is almost no incentive to defect, so in the strategy updating consideration $$C$$’s and $$D$$’s are, holding everything else equal, almost indistinguishable. However, in rewiring an edge the $$C$$ node drops its $$D$$ neighbour, reducing the number of partners for $$D$$, and over time the $$D$$ nodes experience lower payoffs because the numbers of their neighbours decrease. This decrease in degree results in lower total payoff, increasing the probability of a $$D$$ node changing to a $$C$$ strategy, leading to the dominance of $$C$$’s for small $$u$$. In contrast, for $$u = 0.8$$, the $$C$$ nodes become more disadvantaged with increased $$w$$ since the payoff of defecting is high, and the final fraction of cooperators decreases to zero around $$w \doteq 0.1$$. The case $$u = 0.2$$ is intermediate between the competing effects of these two extremes. The final fraction of cooperators increases for values of $$w$$ that are not too large due to the above-described rewiring effect. But sufficiently large strategy update rates (i.e. fewer rewiring steps) combined with an apparently sufficient incentive to defect leads to final network states dominated by defectors. In the simulations, this transition for $$u=0.2$$ appears to occur near $$w\doteq 0.4$$. While the PA and AME results qualitatively describe all of these effects well, AME does a better job capturing this transition point. Taking these results together, we note that in general the increased frequency of breaking and rewiring $$CD$$ bonds (i.e. smaller $$w$$) can directly support cooperation, in agreement with [36, 37], as observed here both in terms of the apparent phase transition and the behaviour at large $$u$$ in the right panel of Fig. 6. But we also note that the results include situations where increasing this frequency (decreasing $$w$$) decreases the total level of cooperation, because of the complex interplay in the model dynamics. 4.2. Network dynamics Having explored the populations in the final states, we consider how the networks evolve to those states. Specifically, we investigate the evolution of five fundamental quantities of the networks: the fractions of nodes in states $$C$$ and $$D$$, and the fractions of edges that are $$CC$$, $$CD$$ and $$DD$$. We recall that the total numbers of nodes and edges are constant during the dynamics, requiring both $$C+D$$ fractions and $$CC+CD+DD$$ fractions to be equal to $$1$$. We visualize the average trajectories through this phase space for a collection of different $$(u,w)$$ parameters in Fig. 7, starting from an initial fraction of defectors $$\rho=0.5$$ distributed across an Erdős–Rényi $$G(N, M)$$ random graph, comparing simulations with PA and AME predictions. As such, the initial fraction of $$CC$$, $$DD$$ and $$CD$$ edges is $$0.25$$, $$0.25$$ and $$0.5$$, respectively; that is, all networks start at $$C=0.5$$, $$CC=0.25$$, $$CD=0.5$$. As the networks evolve, these fractions change, tracing out trajectories in the $$(C,CD)$$ and $$(CC,CD)$$ coordinates in the figure. We perform 50 simulations at each set of parameter values and compute the averages across simulations at each time step. The networks evolve until there are no discordant edges (i.e. $$CD = 0$$). Fig. 7. View largeDownload slide (Color online) Phase space visualizations of the temporal evolution of the fraction of edges that are $$CD$$ versus the fraction of nodes that are $$C$$ (left column) and versus the fraction of edges that are $$CC$$ (right column) for different values of the cost-benefit ratio $$u$$ and strategy updating probability $$w$$ (rows): (a and b) $$u = 0.2$$, $$w = 0.1$$, (c and d) $$u = 0.2$$, $$w = 0.3$$, (e and f) $$u = 0.5$$, $$w = 0.1$$, and (g and h) $$u = 0.5$$, $$w = 0.3$$. Solid lines are the averages of 50 simulations results, dash-dot lines are the pair approximation (PA), and dashed lines are the semi-analytical results of approximate master equations (AME). All networks start at $$C=0.5$$, $$CC=0.25$$ and $$CD=0.5$$. As the system evolves, on average the number of $$CD$$ decreases over time until none of these discordant edges remain ($$CD=0$$). Sufficiently far from the phase transition (as in the top and bottom rows), both approximations are reasonably good. Near the phase transition there are ranges of parameters where only $$AME$$ provides a good description (such as in the second row, where $$PA$$ incorrectly predicts the position relative to the phase transition; compare with the markers and lines in the right panel of Fig. 6). Also near the phase transition, there are other parameters where neither approximation captures the qualitative behaviour (as in the third row, with $$u=0.5$$ and $$w=0.1$$; compare with the location of the observed and predicted phase transition in Fig. 5 and with the markers and lines in the left panel of Fig. 6). Fig. 7. View largeDownload slide (Color online) Phase space visualizations of the temporal evolution of the fraction of edges that are $$CD$$ versus the fraction of nodes that are $$C$$ (left column) and versus the fraction of edges that are $$CC$$ (right column) for different values of the cost-benefit ratio $$u$$ and strategy updating probability $$w$$ (rows): (a and b) $$u = 0.2$$, $$w = 0.1$$, (c and d) $$u = 0.2$$, $$w = 0.3$$, (e and f) $$u = 0.5$$, $$w = 0.1$$, and (g and h) $$u = 0.5$$, $$w = 0.3$$. Solid lines are the averages of 50 simulations results, dash-dot lines are the pair approximation (PA), and dashed lines are the semi-analytical results of approximate master equations (AME). All networks start at $$C=0.5$$, $$CC=0.25$$ and $$CD=0.5$$. As the system evolves, on average the number of $$CD$$ decreases over time until none of these discordant edges remain ($$CD=0$$). Sufficiently far from the phase transition (as in the top and bottom rows), both approximations are reasonably good. Near the phase transition there are ranges of parameters where only $$AME$$ provides a good description (such as in the second row, where $$PA$$ incorrectly predicts the position relative to the phase transition; compare with the markers and lines in the right panel of Fig. 6). Also near the phase transition, there are other parameters where neither approximation captures the qualitative behaviour (as in the third row, with $$u=0.5$$ and $$w=0.1$$; compare with the location of the observed and predicted phase transition in Fig. 5 and with the markers and lines in the left panel of Fig. 6). For comparison, we include both PA and AME in the figure. While some features of the trajectories are better captured than others, both methods in general do reasonably well qualitatively, with better overall accuracy for AME, as seen in the figure. We note that both approximations are qualitatively incorrect for the case $$u=0.5$$, $$w=0.1$$. Recalling the simulation and AME results in the Fig. 5 phase diagram, we note that this parameter set is within the relatively narrow range between the observed location of the phase transition and the AME prediction. 4.3. Degree distributions The AME method utilizes more information and has higher computational cost than PA, but as we have seen it generally provides a better approximation for the current model. Moreover, and unlike PA, AME explicitly includes information about degree distributions. In Fig. 8, we plot the final state degree distributions of $$C$$ and $$D$$ nodes, comparing the simulations and AME predictions. In each case, since we initialize with random strategies on the Erdős–Rényi $$G(N, M)$$ random graph model, the initial degree distributions are approximately Poisson (for large $$N$$). We examine degree distributions in the final states, revisiting the same parameters considered previously: $$u = 0.2, w = 0.1$$; $$u = 0.2, w = 0.3$$; $$u = 0.5, w = 0.1$$; and $$u = 0.5, w = 0.3$$. In the case $$u = 0.5, w = 0.3$$, it is perhaps not surprisingly that AME gives an excellent prediction of the final degree distribution, since we observed excellent prediction of the trajectory of the dynamics in this case above (Fig. 7). In the final state of this case, $$D$$ nodes dominate the whole network, with no $$C$$ nodes left. Fig. 8. View largeDownload slide (Color online) Degree distributions in the stationary states for different values of the cost-benefit ratio $$u$$ and strategy updating probability $$w$$: (a) $$u = 0.2$$, $$w = 0.1$$; (b) $$u = 0.2$$, $$w = 0.3$$; (c) $$u = 0.5$$, $$w = 0.1$$; and (d) $$u = 0.5$$, $$w = 0.3$$. Bars indicate averages from 50 simulations, with colours distinguishing the degree distributions of cooperator and defector nodes. Lines indicate the prediction from the semi-analytical approximate master equations (AME). We note the qualitatively good agreement of the AME prediction with simulations, except in case (c) where AME predicts extinction of the cooperator nodes, as we have previously seen (see Fig. 6(e and f)). Fig. 8. View largeDownload slide (Color online) Degree distributions in the stationary states for different values of the cost-benefit ratio $$u$$ and strategy updating probability $$w$$: (a) $$u = 0.2$$, $$w = 0.1$$; (b) $$u = 0.2$$, $$w = 0.3$$; (c) $$u = 0.5$$, $$w = 0.1$$; and (d) $$u = 0.5$$, $$w = 0.3$$. Bars indicate averages from 50 simulations, with colours distinguishing the degree distributions of cooperator and defector nodes. Lines indicate the prediction from the semi-analytical approximate master equations (AME). We note the qualitatively good agreement of the AME prediction with simulations, except in case (c) where AME predicts extinction of the cooperator nodes, as we have previously seen (see Fig. 6(e and f)). In the other three cases, although the AME predictions are not as accurate, we can still see the AME provides a qualitative picture of the final degree distribution. Once again, the worst case of the four parameter sets visualized here is the $$u=0.5$$, $$w=0.1$$, where the AME prediction is on the wrong side of the phase transition. 4.4. The effect of the initial fraction of defectors, $$\rho$$ In Fig. 9, we explore the effect of the initial fraction of defectors, $$\rho$$. In this figure, we fix the parameter to be $$w = 0.1$$ in the left panel and $$u = 0.2$$ in the right panel, changing the other parameter ($$u$$ and $$w$$, respectively) to view the final fraction of cooperators against $$\rho$$ in stationary states, comparing simulation results (markers) with our AME approximation (lines). By construction, all results in these panels should connect the upper left and lower right corners because when $$\rho = 0$$ (respectively, $$\rho = 1$$) there are no $$D$$ ($$C$$) nodes, no discordant edges, and no dynamics. Fig. 9. View largeDownload slide (Color online) (Left) Fraction of cooperators remaining in the stationary state versus the initial fraction of defectors $$\rho$$, for strategy updating probability $$w = 0.1$$ and various cost-benefit ratios $$u$$: 0, 0.1, ..., 1. (Right) Fraction of cooperators versus $$\rho$$ for $$u = 0.2$$ and various strategy updating probabilities $$w$$: 0, 0.1, ..., 1. In both figures, markers are the averages from 50 simulations and the lines are the semi-analytical predictions from approximate master equations (AME). Fig. 9. View largeDownload slide (Color online) (Left) Fraction of cooperators remaining in the stationary state versus the initial fraction of defectors $$\rho$$, for strategy updating probability $$w = 0.1$$ and various cost-benefit ratios $$u$$: 0, 0.1, ..., 1. (Right) Fraction of cooperators versus $$\rho$$ for $$u = 0.2$$ and various strategy updating probabilities $$w$$: 0, 0.1, ..., 1. In both figures, markers are the averages from 50 simulations and the lines are the semi-analytical predictions from approximate master equations (AME). In the left panel, for the different $$u$$ curves, the final fraction of cooperators first gradually decreases and then drops to zero suddenly at different values of $$\rho$$. AME predicts this qualitative behaviour for different levels of $$u$$. Moreover, fixing $$w = 0.1$$, smaller $$u$$ results in higher cooperation, since the incentive of defecting is less when $$u$$ is small. In the right panel, fixing $$u = 0.2$$, the diagonal line is the case $$w = 0$$, where there are no strategy updates and the cooperation level remains the same during the evolution. For small and moderate $$w$$, the curves decrease with increasing $$\rho$$ (though more quickly for smaller $$w$$), followed by a sharp drop to zero at decreasing values of $$\rho$$ for increased $$w$$. When $$w$$ is large (greater rates of strategy updates), the final cooperative level drops to zero for all non-zero $$\rho$$. As we show in this panel, the AME curves suddenly drop to zero when $$w \geq 0.4$$. 5. A model variant with $$\boldsymbol{DD}$$ rewiring In this section, we study a slightly different variant of the partner switching evolution game model, also introduced by [19], where $$DD$$ edges can also rewire as defector nodes seek out new partners that they might better exploit. We are particularly interested in how this additional rewiring affects the final cooperation level. In this variant, each time step starts by uniformly at random selecting from the union of $$CD$$ and $$DD$$ edges. (Recall that only $$CD$$ edges were selected in the original model above.) If a $$CD$$ edge is picked, the dynamics proceed as in the original model: with probability $$w$$, the strategy update process happens; otherwise (that is, with probability $$1-w$$) the $$C$$ node along this edge unilaterally drops the partnership with its $$D$$ neighbour and rewires to another node. In contrast, if a $$DD$$ edge is picked, then with probability $$w$$ nothing happens (there is no strategy update); otherwise (that is, with probability $$1-w$$) one of the two $$D$$ ends, selected with equal probability, drops the connection and rewires uniformly at random to another node in the network to whom it is not already connected. We note that the total numbers of nodes, $$N$$, and edges, $$M$$, remain constant, as in the original model. This variant model stops evolving when there are no $$CD$$ or $$DD$$ edges remaining in the system; that is, all edges in a final, frozen state are $$CC$$. However, this does not require that all $$D$$ nodes are removed from the system, only that they are each left isolated with no connections. Another possibility, in which the model variant as stated above never reaches a stopping condition, is that the system evolves to a statistically stationary state where all $$C$$ nodes have been removed so that all edges are $$DD$$ edges and the continued rewiring of these $$DD$$ edges simply serves to repeatedly randomize the network of defector nodes. Since in this case there are no $$C$$ nodes left to emulate in a strategy update nor to connect to in a rewiring, we do not consider the dynamics further beyond this event. Both of these cases are visualized in Fig. 10. In the left panel, the $$C$$ nodes own all of the edges, having isolated the exploitation efforts of the $$D$$ nodes. In the right panel, only $$D$$ nodes remain and all edges are $$DD$$. Fig. 10. View largeDownload slide (Color online) Visualization of stationary states of the model variant with $$DD$$ rewiring, for $$N \,{=}\, 1,000$$ nodes, $$M \,{=}\, 5,000$$ edges, and initial fraction of defectors $$\rho \,{=}\, 0.5$$, for different parameters. Colours correspond to the two node states: cooperating (light/blue) and defecting (dark/red). (Left) For cost-benefit ratio $$u \,{=}\, 0.5$$ and strategy updating probability $$w \,{=}\, 0.1$$, $$D$$ nodes remain in the system but they have each been isolated with zero degree. (Right) For $$u \,{=}\, 1$$ and $$w \,{=}\, 0.5$$, the system evolves until only $$D$$ nodes exist in the network and all the edges are $$DD$$ edges. These visualizations were created using the Yifan Hu layout in Gephi. Fig. 10. View largeDownload slide (Color online) Visualization of stationary states of the model variant with $$DD$$ rewiring, for $$N \,{=}\, 1,000$$ nodes, $$M \,{=}\, 5,000$$ edges, and initial fraction of defectors $$\rho \,{=}\, 0.5$$, for different parameters. Colours correspond to the two node states: cooperating (light/blue) and defecting (dark/red). (Left) For cost-benefit ratio $$u \,{=}\, 0.5$$ and strategy updating probability $$w \,{=}\, 0.1$$, $$D$$ nodes remain in the system but they have each been isolated with zero degree. (Right) For $$u \,{=}\, 1$$ and $$w \,{=}\, 0.5$$, the system evolves until only $$D$$ nodes exist in the network and all the edges are $$DD$$ edges. These visualizations were created using the Yifan Hu layout in Gephi. 5.1. Semi-analytical methods of approximation As before, we study this model variant with a combination of simulations and approximate analytic models, comparing the PA and AME approaches with simulation results. Following the notation of the original model in Section 3, the PA equations for this variant become   dNCdt= w⋅NCD⋅tanh⁡[α2(π¯C−π¯D)]dNCCdt= w⋅(NCDϕC→D−2NCDNCCNCϕD→C+NCDNCDNDϕC→D) +(1−w)⋅NCNNCDdNDDdt= w⋅(NCDϕD→C−2NCDNDDNDϕC→D+NCDNCDNCϕD→C) −(1−w)⋅NCNNDD (5.1) where we note the only difference between equation (5.1) and equation (3.1) is the appearance of the last term in the $$dN_{DD}/dt$$ equation. This term captures the possibility in this variant model for a $$D$$ node to dismiss its defective partner and rewire to a $$C$$ node, thus decreasing the $$DD$$ count. Similarly, we modify the AME equations to account for the rewiring of $$DD$$ edges, yielding   dCk,ldt =w{ϕk,lD⋅(k−l)Dk,l−ϕk,lC⋅lCk,l +ϕCD←CC⋅γC(l+1)Ck,l+1−ϕCD←CC⋅γClCk,l +ϕCC←CD⋅βC(k−l+1)Ck,l−1−ϕCC←CD⋅βC(k−l)Ck,l} +(1−w){NCN[(l+1)Ck,l+1−lCk,l]+NCDN[Ck−1,l−Ck,l] +NDDN[Ck−1,l−1−Ck,l]} (5.2)  dDk,ldt =w{−ϕk,lD⋅(k−l)Dk,l+ϕk,lC⋅lCk,l +ϕDD←DC⋅γD(l+1)Dk,l+1−ϕDD←DC⋅γDlDk,l +ϕDC←DD⋅βD(k−l+1)Dk,l−1−ϕDC←DD⋅βD(k−l)Dk,l} +(1−w){[(k−l+1)Dk+1,l−(k−l)Dk,l]+NCDN[Dk−1,l−Dk,l] +NC2N[(l+1)Dk,l+1−lDk,l]+NDDN[Dk−1,l−1−Dk,l] +12[(l+1)Dk+1,l+1−lDk,l]}. (5.3) Again, the additional terms here compared to equations (3.2) and (3.3) account for the rewiring of $$DD$$ edges, accounting for the rates of $$D$$ nodes abandoning $$D$$ neighbours and rewiring to $$C$$ nodes, $$D$$ nodes passively gaining new connections through this rewiring, and $$D$$ nodes being abandoned by their $$D$$ neighbours. Note the $$1/2$$ factors in the last two lines of the $$D_{k,l}$$ equation, accounting for an individual node on a $$DD$$ keeping and losing the edge. 5.2. Final level of cooperation As before, we compare the final fraction of cooperators in the simulations versus the PA and AME predictions. In the left panel of Fig. 11, we consider $$w = 0$$, $$w = 0.05$$, $$w = 0.1$$, and $$w = 0.5$$, plotting the fraction of $$C$$ nodes in the final states versus the cost-benefit ratio, $$u$$. In the right panel, we fix values of $$u$$ and vary $$w$$. Again, markers indicate simulation results, dotted lines are from the PA equations, and dashed lines are from AME. Fig. 11. View largeDownload slide (Color online) (Left) Fraction of cooperators remaining in the stationary state of the model variant with $$DD$$ rewiring versus cost-benefit ratio $$u$$, for different strategy updating probabilities $$w$$. Markers are averages from 1,000 simulations, dotted lines are the semi-analytical results from pair approximation (PA), and the dashed lines are the semi-analytical results from approximate master equations (AME). (Right) Fraction of cooperators remaining in the stationary state versus $$w$$ for different $$u$$ values, with markers and lines as in the left panel. Fig. 11. View largeDownload slide (Color online) (Left) Fraction of cooperators remaining in the stationary state of the model variant with $$DD$$ rewiring versus cost-benefit ratio $$u$$, for different strategy updating probabilities $$w$$. Markers are averages from 1,000 simulations, dotted lines are the semi-analytical results from pair approximation (PA), and the dashed lines are the semi-analytical results from approximate master equations (AME). (Right) Fraction of cooperators remaining in the stationary state versus $$w$$ for different $$u$$ values, with markers and lines as in the left panel. Comparing Fig. 11 for this variant with the results in Fig. 6, the inclusion of $$DD$$ rewiring here leads to slightly larger final fractions of $$C$$ nodes for small $$w>0$$. In particular, whereas an existing $$DD$$ link in the original model can only be changed if the state of one of the two $$D$$ nodes changes in a strategy update, the $$DD$$ rewiring in this variant increases the numbers of $$CD$$ edges and thus leaves open the possibility of $$C$$ nodes becoming more favourable because of their yet higher degree. However, for larger values of $$w$$, the more rapid rate of strategy updating leads to defectors dominating the system before the cooperators can gain any advantage from increased degrees. For further comparison, we consider the final fractions of $$C$$ nodes and $$CC$$ edges for different combinations of $$u$$ and $$w$$ in Fig. 12, as compared with the similar plots in Fig. 5. While many of the general features remain the same in the presence of this DD-rewiring variant, a notable difference is in the final fraction of $$CC$$ edges for small $$w$$. In particular, at $$w=0$$ (no strategy updates), the original model results visualized in Fig. 5 limit to a final $$CC$$ level of 0.75, since the initial $$DD$$ edges (at 0.25 for $$\rho=0.5$$) cannot ever rewire under these settings. But these edges of course do rewire in the $$DD$$-rewiring variant, leading $$CC$$ to limit to 1 as $$w\to 0$$. Fig. 12. View largeDownload slide (Color online) Results from simulations (left column) and approximate master equations (AME) (right column) of the final fractions of $$C$$ nodes (top row) and $$CC$$ links (bottom row) for different combinations of the cost-benefit ratio $$u$$ and strategy updating probability $$w$$ in the model variant with $$DD$$ rewiring. The initial fraction of defectors is $$\rho=0.5$$. For both the $$u$$ and $$w$$ axes, we use steps of 0.05 and plot the results from stationary states. Simulation results here are averaged over 50 realizations at each parameter set. These visualizations were generated from results on a regular grid through bilinear interpolation, leading to some clearly apparent grid artefacts. As in the original model without $$DD$$ rewiring (Fig. 5), while some discrepancies between simulation and AME results are clearly present, we note in particular that the position of the phase transition in the $$(u,w)$$ parameter space is well approximated by the AME system. Fig. 12. View largeDownload slide (Color online) Results from simulations (left column) and approximate master equations (AME) (right column) of the final fractions of $$C$$ nodes (top row) and $$CC$$ links (bottom row) for different combinations of the cost-benefit ratio $$u$$ and strategy updating probability $$w$$ in the model variant with $$DD$$ rewiring. The initial fraction of defectors is $$\rho=0.5$$. For both the $$u$$ and $$w$$ axes, we use steps of 0.05 and plot the results from stationary states. Simulation results here are averaged over 50 realizations at each parameter set. These visualizations were generated from results on a regular grid through bilinear interpolation, leading to some clearly apparent grid artefacts. As in the original model without $$DD$$ rewiring (Fig. 5), while some discrepancies between simulation and AME results are clearly present, we note in particular that the position of the phase transition in the $$(u,w)$$ parameter space is well approximated by the AME system. 6. Conclusion We have investigated a node-based prisoner’s dilemma game played on a network coevolving with player strategy updates. Our study includes two model variants for rewiring: one where only $$CD$$ edges can rewire, by $$C$$ nodes dropping a link to a $$D$$ neighbour in favour of a new partner; and a variant where $$DD$$ edges can also rewire. We explore the parameter space to investigate the competing effects of strategy updates and partner switching, as well as the initial levels of cooperation versus defection. We compare our simulations to the existing PA developed in [19], and we develop an approximation using AME to more accurately capture the transitions between the properties of the final states of these models. We also use the AME method to estimate the final-state degree distributions for different parameters. We are particularly interested in the features that determine the final level of cooperation and overall utility in the network. Revisiting Fig. 6, and the corresponding results for the model variant in Fig. 11, it is clear that the most effective way to increase the final fraction of cooperators is to directly reduce the cost-benefit ratio, $$u$$, so that players do not have as much incentive to defect. Alternatively, if the strategy update rate $$w$$ is small enough, the effects of rewiring give greater advantages to $$C$$ nodes as they accumulate larger numbers of playing partners, which then results in larger numbers of cooperators through the strategy updates. Of course, one can also directly decrease the initial fraction of defectors. The question of how to maximize the total utility or payoff relates to the types of edges. In the partner-switching evolutionary game model we study here, the total number of nodes and edges remain fixed, no matter how the networks and node states change. Utility comes from every edge in the network, with each $$CC$$, $$CD$$ and $$DD$$ edge contributing (from the two nodes) $$2$$, $$1+u$$, and $$2u$$, respectively, where $$u \in (0,1)$$. With no $$CD$$ edges in the final state, maximizing the final overall utility reduces to maximizing the number of $$CC$$ links. In particular, comparing the right panels of Figs 5 and 12 for the two model variants, whether or not $$DD$$ edges can rewire significantly affects the $$CC$$ count (and thus the total system utility) for small strategy update rates. Acknowledgments We are grateful to the reviewers for identifying multiple important references cited here. Funding Research reported in this publication was supported by the Eunice Kennedy Shriver National Institute of Child Health and Human Development of the National Institutes of Health under Award Number R01HD075712. The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institutes of Health. References 1. Gibbons, R. (1992) A Primer in Game Theory . Coleshill, Birmingham B46 1EE, UK: Harvester Wheatsheaf. 2. Harsanyi, J. C., Selten, R. (1988) A General Theory of Equilibrium Selection in Games , vol. 1. Cambridge, MA 02142, USA: MIT Press Books. 3. Myerson, R. B. (1991) Game Theory: Analysis of Conflict . Cambridge, MA: Harvard University Press. 4. Hauert, C., Holmes, M. & Doebeli, M. (2006) Evolutionary games and population dynamics: maintenance of cooperation in public goods games. Proc. R. Soc. Lond B Biol. Sci. , 273, 2565– 2571. Google Scholar CrossRef Search ADS   5. Hofbauer, J. & Sigmund, K. (2003) Evolutionary game dynamics. Bull. Am. Math. Soc. , 40, (4), 479– 519. Google Scholar CrossRef Search ADS   6. Nowak, M. A. & May, R. M. (1993) The spatial dilemmas of evolution. Int. J. Bifurcat. Chaos , 3, (01), 35– 78. Google Scholar CrossRef Search ADS   7. Smith, J. M. (1982) Evolution and the Theory of Games . Cambridge CB2 8BS, UK: Cambridge University Press. 8. Weibull, J. W. (1997) Evolutionary Game Theory . Cambridge, MA 02142, USA: MIT Press. 9. Mitteldorf, J. & Wilson, D. S. (2000) Population viscosity and the evolution of altruism. J. Theoret. Biol. , 204, (4), 481– 496. Google Scholar CrossRef Search ADS   10. Fletcher, J. A. & Zwick, M. (2007) The evolution of altruism: game theory in multilevel selection and inclusive fitness. J. Theoret. Biol. , 245, (1), 26– 36. Google Scholar CrossRef Search ADS   11. Axelrod, R. (2006) The Evolution of Cooperation , revised edn. New York, NY 10104, USA: Basic Books. 12. Miller, J. H. (1996) The coevolution of automata in the repeated prisoner’s dilemma. J. Econ. Behav. Organ. , 29, (1), 87– 112. Google Scholar CrossRef Search ADS   13. Mertens, J.-F. (1989) Repeated Games . New York, NY 10036, USA: Springer. 14. Fowler, J. H. & Laver, M. (2008) A tournament of party decision rules. J. Conflict Resolut. , 52, (1), 68– 92. Google Scholar CrossRef Search ADS   15. Nisan, N., Roughgarden, T., Tardos, E. & Vazirani, V. V. (2007) Algorithmic Game Theory , vol. 1. Cambridge: Cambridge University Press. 16. Abramson, G. & Kuperman, M. (2001) Social games in a social network. Phys. Rev. E , 63, (3), 030901. Google Scholar CrossRef Search ADS   17. Ebel, H. & Bornholdt, S. (2002) Coevolutionary games on networks. Phys. Rev. E , 66, (5), 056118. Google Scholar CrossRef Search ADS   18. Fu, F., Hauert, C., Nowak, M. A. & Wang, L. (2008) Reputation-based partner choice promotes cooperation in social networks. Phys. Rev. E , 78, (2), 026117. Google Scholar CrossRef Search ADS   19. Fu, F., Wu, T. & Wang, L. (2009) Partner switching stabilizes cooperation in coevolutionary prisoner’s dilemma. Phys. Rev. E , 79, (3), 036101. Google Scholar CrossRef Search ADS   20. Hauert, C. & Doebeli, M. (2004) Spatial structure often inhibits the evolution of cooperation in the snowdrift game. Nature , 428, (6983), 643– 646. Google Scholar CrossRef Search ADS   21. Hidalgo, J., Grilli, J., Suweis, S., Maritan, A. & Muñoz, M. A. (2015) Cooperation, competition and the emergence of criticality in communities of adaptive systems. J. Stat. Mech.: Theor. Experi.,  2016, (3), 033203. 22. Jackson, M. O. & Watts, A. (2002) On the formation of interaction networks in social coordination games. Games Econ. Behav. , 41, (2), 265– 291. Google Scholar CrossRef Search ADS   23. Killingback, T. & Doebeli, M. (1996) Spatial evolutionary game theory: Hawks and Doves revisited. Proc. R. Soc. Lond. B Biol. Sci. , 263, (1374), 1135– 1144. Google Scholar CrossRef Search ADS   24. Lieberman, E., Hauert, C. & Nowak, M. A. (2005) Evolutionary dynamics on graphs. Nature , 433, (7023), 312– 316. Google Scholar CrossRef Search ADS   25. Nowak, M. A. & May, R. M. (1992) Evolutionary games and spatial chaos. Nature , 359, (6398), 826– 829. Google Scholar CrossRef Search ADS   26. Kerr, B., Riley, M. A., Feldman, M. W. & Bohannan, B. J. (2002) Local dispersal promotes biodiversity in a real-life game of rock–paper–scissors. Nature , 418, (6894), 171– 174. Google Scholar CrossRef Search ADS   27. Pinheiro, F. L., Santos, F. C. & Pacheco, J. M. (2016) Linking individual and collective behavior in adaptive social networks. Phys. Rev. Lett. , 116, (12), 128702. Google Scholar CrossRef Search ADS   28. Santos, F. C., Pacheco, J. M. & Lenaerts, T. (2006) Cooperation prevails when individuals adjust their social ties. PLoS Comput. Biol. , 2, (10), e140. Google Scholar CrossRef Search ADS   29. Hauert, C., De Monte, S., Hofbauer, J. & Sigmund, K. (2002) Replicator dynamics for optional public good games. J. Theoret. Biol. , 218, (2), 187– 194. Google Scholar CrossRef Search ADS   30. Perc, M., Gómez-Gardeñes, J., Szolnoki, A., Floría, L. M. & Moreno, Y. (2013) Evolutionary dynamics of group interactions on structured populations: a review. J. R. Soc. Interface , 10, (80), 20120997. Google Scholar CrossRef Search ADS   31. Szolnoki, A., Perc, M. & Szabó, G. (2009) Topology-independent impact of noise on cooperation in spatial public goods games. Phys. Rev. E , 80, (5), 056109. Google Scholar CrossRef Search ADS   32. Pacheco, J. M., Traulsen, A. & Nowak, M. A. (2006) Active linking in evolutionary games. J. Theoret. Biol. , 243, (3), 437– 443. Google Scholar CrossRef Search ADS   33. Pacheco, J. M., Traulsen, A. & Nowak, M. A. (2006) Coevolution of strategy and structure in complex networks with dynamical linking. Phys. Rev. Lett. , 97, (25), 258103. Google Scholar CrossRef Search ADS   34. Perc, M. & Szolnoki, A. (2010) Coevolutionary games—a mini review. BioSystems , 99, (2), 109– 125. Google Scholar CrossRef Search ADS   35. Szolnoki, A. & Perc, M. (2009) Emergence of multilevel selection in the prisoner’s dilemma game on coevolving random networks. New J. Phys. , 11, (9), 093033. Google Scholar CrossRef Search ADS   36. Szolnoki, A. & Perc, M. (2009) Resolving social dilemmas on evolving random networks. Europhys. Lett. , 86, (3), 30007. Google Scholar CrossRef Search ADS   37. Szolnoki, A., Perc, M. & Danku, Z. (2008) Making new connections towards cooperation in the prisoner’s dilemma game. Europhys. Lett. , 84, (5), 50007. Google Scholar CrossRef Search ADS   38. Gómez-Gardeñes, J., Campillo, M., Floría, L. & Moreno, Y. (2007) Dynamical organization of cooperation in complex topologies. Phys. Rev. Lett. , 98, (10), 108103. Google Scholar CrossRef Search ADS   39. Perc, M. (2009) Evolution of cooperation on scale-free networks subject to error and attack. New J. Phys. , 11, (3), 033027. Google Scholar CrossRef Search ADS   40. Pusch, A., Weber, S. & Porto, M. (2008) Impact of topology on the dynamical organization of cooperation in the prisoner’s dilemma game. Phys. Rev. E , 77, (3), 036120. Google Scholar CrossRef Search ADS   41. Perc, M., Szolnoki, A. & Szabó, G. (2008) Restricted connections among distinguished players support cooperation. Phys. Rev. E , 78, (6), 066101. Google Scholar CrossRef Search ADS   42. Roca, C. P., Cuesta, J. A. & Sánchez, A. (2009) Promotion of cooperation on networks? The myopic best response case. Eur. Phys. J. B , 71, (4), 587– 595. Google Scholar CrossRef Search ADS   43. Kim, B. J., Trusina, A., Holme, P., Minnhagen, P., Chung, J. S. & Choi, M. (2002) Dynamic instabilities induced by asymmetric influence: prisoners’ dilemma game in small-world networks. Phys. Rev. E , 66, (2), 021907. Google Scholar CrossRef Search ADS   44. Santos, F. C., Rodrigues, J. & Pacheco, J. M. (2005) Epidemic spreading and cooperation dynamics on homogeneous small-world networks. Phys. Rev. E , 72, (5), 056128. Google Scholar CrossRef Search ADS   45. Szolnoki, A., Perc, M., Szabó, G. & Stark, H.-U. (2009) Impact of aging on the evolution of cooperation in the spatial prisoner’s dilemma game. Phys. Rev. E , 80, (2), 021901. Google Scholar CrossRef Search ADS   46. Fu, F., Chen, X., Liu, L. & Wang, L. (2007) Social dilemmas in an online social network: the structure and evolution of cooperation. Phys. Lett. A , 371, (1), 58– 64. Google Scholar CrossRef Search ADS   47. Holme, P., Trusina, A., Kim, B. J. & Minnhagen, P. (2003) Prisoners’ dilemma in real-world acquaintance networks: Spikes and quasiequilibria induced by the interplay between structure and dynamics. Phys. Rev. E , 68, (3), 030901. Google Scholar CrossRef Search ADS   48. Lozano, S., Arenas, A. & Sánchez, A. (2008) Mesoscopic structure conditions the emergence of cooperation on social networks. PLoS ONE , 3, (4), e1892. Google Scholar CrossRef Search ADS   49. Wang, Z., Wang, L., Szolnoki, A. & Perc, M. (2015) Evolutionary games on multilayer networks: a colloquium. Eur. Phys. J. B , 88, (5), 1– 15. 50. Hamilton, W. (1964) The genetical evolution of social behaviour. I. J. Theoret. Biol. , 7, 1– 16. Google Scholar CrossRef Search ADS   51. Dugatkin, L. A. & Mesterton-Gibbons, M. (1996) Cooperation among unrelated individuals: reciprocal altruism, by-product mutualism and group selection in fishes. BioSystems , 37, (1), 19– 30. Google Scholar CrossRef Search ADS   52. Traulsen, A., Röhl, T. & Schuster, H. G. (2004) Stochastic gain in population dynamics. Phys. Rev. Lett. , 93, (2), 028701. Google Scholar CrossRef Search ADS   53. Pacheco, J. M., Traulsen, A., Ohtsuki, H. & Nowak, M. A. (2008) Repeated games and direct reciprocity under active linking. J. Theoret. Biol. , 250, (4), 723– 731. Google Scholar CrossRef Search ADS   54. Nowak, M. A. & Sigmund, K. (1998) The dynamics of indirect reciprocity. J. Theoret. Biol. , 194, (4), 561– 574. Google Scholar CrossRef Search ADS   55. Nowak, M. A. & Sigmund, K. (2005) Evolution of indirect reciprocity. Nature , 437, (7063), 1291– 1298. Google Scholar CrossRef Search ADS   56. Perc, M. & Szolnoki, A. (2008) Social diversity and promotion of cooperation in the spatial prisoner’s dilemma game. Phys. Rev. E , 77, (1), 011904. Google Scholar CrossRef Search ADS   57. Santos, F. C., Santos, M. D. & Pacheco, J. M. (2008) Social diversity promotes the emergence of cooperation in public goods games. Nature , 454, (7201), 213– 216. Google Scholar CrossRef Search ADS   58. Yang, H.-X., Wang, W.-X., Wu, Z.-X., Lai, Y.-C. & Wang, B.-H. (2009) Diversity-optimized cooperation on complex networks. Phys. Rev. E , 79, (5), 056107. Google Scholar CrossRef Search ADS   59. Chen, Y., Qin, S.-M., Yu, L., Zhang, S. (2008) Emergence of synchronization induced by the interplay between two prisoner’s dilemma games with volunteering in small-world networks. Phys. Rev. E , 77, (3), 032103. Google Scholar CrossRef Search ADS   60. Hauert, C., De Monte, S., Hofbauer, J. & Sigmund, K. (2002) Volunteering as red queen mechanism for cooperation in public goods games. Science , 296, (5570), 1129– 1132. Google Scholar CrossRef Search ADS   61. Hauert, C., Traulsen, A., Brandt, H., Nowak, M. A. & Sigmund, K. (2007) Via freedom to coercion: the emergence of costly punishment. Science , 316, (5833), 1905– 1907. Google Scholar CrossRef Search ADS   62. Szabó, G. & Tőke, C. (1998) Evolutionary prisoner’s dilemma game on a square lattice. Phys. Rev. E , 58, (1), 69. Google Scholar CrossRef Search ADS   63. Fudenberg, D., Nowak, M. A., Taylor, C. & Imhof, L. A. (2006) Evolutionary game dynamics in finite populations with strong selection and weak mutation. Theoret. Popul. Biol. , 70, (3), 352– 363. Google Scholar CrossRef Search ADS   64. Traulsen, A., Nowak, M. A. & Pacheco, J. M. (2007) Stochastic payoff evaluation increases the temperature of selection. J. Theoret. Biol. , 244, (2), 349– 356. Google Scholar CrossRef Search ADS   65. Bastian, M., Heymann, S., Jacomy, M. (2009) Gephi: an open source software for exploring and manipulating networks.. ICWSM , 8, 361– 362. 66. Ren, J., Wang, W.-X. & Qi, F. (2007) Randomness enhances cooperation: a resonance-type phenomenon in evolutionary games. Phys. Rev. E , 75, (4), 045101. Google Scholar CrossRef Search ADS   67. Vukov, J., Szabó, G. & Szolnoki, A. (2006) Cooperation in the noisy case: Prisoner’s dilemma game on two types of regular random graphs. Phys. Rev. E , 73, (6), 067103. Google Scholar CrossRef Search ADS   68. Khoo, T., Fu, F. & Pauls, S. (2016) Coevolution of cooperation and partner rewiring range in spatial social networks. Sci. Rep. , 6, 36293. Google Scholar CrossRef Search ADS PubMed  69. Gleeson, J. P. (2013) Binary-state dynamics on complex networks: pair approximation and beyond. Phys. Rev. X , 3, (2), 021004. 70. Zhou, J., Xiao, G. & Chen, G. (2013) Link-based formalism for time evolution of adaptive networks. Phys. Rev. E , 88, (3), 032808. Google Scholar CrossRef Search ADS   © The authors 2017. Published by Oxford University Press. All rights reserved.

### Journal

Journal of Complex NetworksOxford University Press

Published: Feb 1, 2018

## You’re reading a free preview. Subscribe to read the entire article.

### DeepDyve is your personal research library

It’s your single place to instantly
that matters to you.

over 18 million articles from more than
15,000 peer-reviewed journals.

All for just $49/month ### Explore the DeepDyve Library ### Search Query the DeepDyve database, plus search all of PubMed and Google Scholar seamlessly ### Organize Save any article or search result from DeepDyve, PubMed, and Google Scholar... all in one place. ### Access Get unlimited, online access to over 18 million full-text articles from more than 15,000 scientific journals. ### Your journals are on DeepDyve Read from thousands of the leading scholarly journals from SpringerNature, Elsevier, Wiley-Blackwell, Oxford University Press and more. All the latest content is available, no embargo periods. DeepDyve ### Freelancer DeepDyve ### Pro Price FREE$49/month
\$360/year

Save searches from
PubMed

Create lists to

Export lists, citations