Smale Strategies For The N-Person Iterated Prisoner's Dilemma

Ethan Akin,Slawomir Plaskacz,Joanna Zwierzchowska

DOI: https://doi.org/10.48550/arXiv.1802.03829

2018-02-12

Abstract:Adapting methods introduced by Steven Smale, we describe good strategies for a symmetric version of the Iterated Prisoner's Dilemma with n players.

Dynamical Systems,Computer Science and Game Theory

What problem does this paper attempt to address?

The Iterated Prisoner's Dilemma: Good Strategies and Their Dynamics

Ethan Akin

DOI: https://doi.org/10.48550/arXiv.1211.0969

2017-04-26

Abstract:For the iterated Prisoner's Dilemma, there exist Markov strategies which solve the problem when we restrict attention to the long term average payoff. When used by both players these assure the cooperative payoff for each of them. Neither player can benefit by moving unilaterally any other strategy, i.e. these are Nash equilibria. In addition, if a player uses instead an alternative which decreases the opponent's payoff below the cooperative level, then his own payoff is decreased as well. Thus, if we limit attention to the long term payoff, these \emph{good strategies} effectively stabilize cooperative behavior. We characterize these good strategies and analyze their role in evolutionary dynamics.

Dynamical Systems,Computer Science and Game Theory
Properties of Winning Iterated Prisoner's Dilemma Strategies

Nikoleta E. Glynatsi,Vincent Knight,Marc Harper

2024-01-24

Abstract:Researchers have explored the performance of Iterated Prisoner's Dilemma strategies for decades, from the celebrated performance of Tit for Tat to the introduction of the zero-determinant strategies and the use of sophisticated learning structures such as neural networks. Many new strategies have been introduced and tested in a variety of tournaments and population dynamics. Typical results in the literature, however, rely on performance against a small number of somewhat arbitrarily selected strategies in a small number of tournaments, casting doubt on the generalizability of conclusions. In this work, we analyze a large collection of 195 strategies in thousands of computer tournaments, present the top performing strategies across multiple tournament types, and distill their salient features. The results show that there is not yet a single strategy that performs well in diverse Iterated Prisoner's Dilemma scenarios, nevertheless there are several properties that heavily influence the best performing strategies. This refines the properties described by Axelrod in light of recent and more diverse opponent populations to: be nice, be provocable and generous, be a little envious, be clever, and adapt to the environment. More precisely, we find that strategies perform best when their probability of cooperation matches the total tournament population's aggregate cooperation probabilities. The features of high performing strategies help cast some light on why strategies such as Tit For Tat performed historically well in tournaments and why zero-determinant strategies typically do not fare well in tournament settings. Furthermore, our findings have implications for the future training of autonomous agents, as understanding the crucial features for incorporation into these agents becomes essential.

Computer Science and Game Theory
Network-Iterated Prisoner's Dilemma

Martín Soto Quintanilla

DOI: https://doi.org/10.48550/arXiv.2211.16329

2022-10-15

Abstract:We introduce the stochastic Network-Iterated Prisoner's Dilemma (NIPD) model, a network of players playing the Prisoner's Dilemma with their neighbours, each with a memory-one strategy which they constantly and locally update to improve their success. This process is non-deterministic, and mirrors societal interactions in many relevant aspects. We use it to assess the flexibility, noise tolerance and real-world adaptability of some well-known strategies. Furthermore, in the model a new strategy naturally emerges which proves way more successful than those. We also derive some theoretical parameters that gauge the success of a strategy in this context.

Physics and Society
Invincible Strategies of Iterated Prisoner's Dilemma

Shiheng Wang,Fangzhen Lin

DOI: https://doi.org/10.48550/arXiv.1712.06488

2017-12-19

Abstract:Iterated Prisoner's Dilemma(IPD) is a well-known benchmark for studying the long term behaviors of rational agents, such as how cooperation can emerge among selfish and unrelated agents that need to co-exist over long term. Many well-known strategies have been studied, from the simple tit-for-tat(TFT) strategy made famous by Axelrod after his influential tournaments to more involved ones like zero determinant strategies studied recently by Press and Dyson. In this paper, following Press and Dyson, we consider one memory probabilistic strategies. We consider that we call invincible strategies: a strategy is invincible if it never loses against any other strategy in terms of average payoff in the limit, if the limit exists. We show a strategy is invincible for infinite repeated iterated prisoner's dilemma.

Computer Science and Game Theory
An Investigation of N-person Prisoners' Dilemmas

,Miklos N. Szilagyi

DOI: https://doi.org/10.25088/complexsystems.14.2.155

2024-06-15

Complex Systems

Abstract:This paper is an attempt to systematically present the problem of various N-person Prisoners' Dilemma games and some of their possible solutions. 13 characteristics of the game are discussed. The role of payoff curves, personalities, and neighborhood is investigated. Experiments are performed with a new simulation tool for multi-agent Prisoners' Dilemma games. Investigations of realistic situations in which agents have various personalities show interesting new results. For the case of pavlovian agents the game has nontrivial but remarkably regular solutions. Examples of nonuniform distributions and mixed personalities are also presented. All solutions strongly depend on the choice of parameter values.
Network Structures between Strategies in Iterated Prisoners' Dilemma Games

Young Jin Kim,Myungkyoon Roh,Seung-Woo Son

DOI: https://doi.org/10.48550/arXiv.1403.1048

2014-03-05

Physics and Society

Abstract:We use replicator dynamics to study an iterated prisoners' dilemma game with memory. In this study, we investigate the characteristics of all 32 possible strategies with a single-step memory by observing the results when each strategy encounters another one. Based on these results, we define similarity measures between the 32 strategies and perform a network analysis of the relationship between the strategies by constructing a strategies network. Interestingly, we find that a win-lose circulation, like rock-paper-scissors, exists between strategies and that the circulation results from one unusual strategy.
Press-Dyson Analysis of Asynchronous, Sequential Prisoner's Dilemma

Robert D. Young

DOI: https://doi.org/10.48550/arXiv.1712.05048

2017-12-12

Abstract:Two-player games have had a long and fruitful history of applications stretching across the social, biological, and physical sciences. Most applications of two-player games assume synchronous decisions or moves even when the games are iterated. But different strategies may emerge as preferred when the decisions or moves are sequential, or the games are iterated. Zero-determinant strategies developed by Press and Dyson are a new class of strategies that have been developed for synchronous two-player games, most notably the iterated prisoner's dilemma. Here we apply the Press-Dyson analysis to sequential or asynchronous two-player games. We focus on the asynchronous prisoner's dilemma. As a first application of the Press-Dyson analysis of the asynchronous prisoner's dilemma, tit-for-tat is shown to be an efficient defense against extortionate zero-determinant strategies. Nice strategies like tit-for-tat are also shown to lead to Pareto optimal payoffs for both players in repeated prisoner's dilemma.

Biological Physics,Computer Science and Game Theory,Physics and Society
Space Modeling of Problem Solving Strategies of “Prisoner's Dilemma”

D.J. Balanev,

DOI: https://doi.org/10.17223/17267080/78/6

2021-01-01

Sibirskiy psikhologicheskiy zhurnal

Abstract:An iterated version of the game "Prisoner's Dilemma" is used as a model of cooperation largely due to the wide range of strategies that the subjects can use. The problem of the effec-tiveness of strategies for solving the Iterated Prisoner's Dilemma (IPD) is most often considered from the point of view of information models, where strategies do not take into account the relationship that arise when real people play. Some of these strategies are obvious, others depend upon social context. In our paper, we use one of the promising directions in the development of studying IPD strategies – the use of artificial neural networks. We use neural networks as a modeling tool and as a part of game environment. The main goal of our work is to build an information model that predicts the behavior of an individual person as well as group of people in the situation of solving of social dilemma. It takes into account social relationship, including those caused by experimental influence, gender differences, and individual differences in the strategy for solving cognitive tasks. The model demonstrates the transition of individual actions into socially determined behavior. Evaluation of the effect of socialization associated with the procedure of the game provides additional information about the effectiveness and characteristics of the experimental impact.The paper defines the minimum unit of analysis of the IPD player's strategy in a group, the identity with which can be considered as a variable. It discusses the influence of the experi-mentally formed group identity on the change of preferred strategies in social dilemmas. We use the possibilities of neural networks as means of categorizing the results of the prisoner's iterative dilemma in terms of the strategy applied by the player, as well as social factors. We define the patterns of changes in the IPD player's strategy before and after socialization are determined. The paper discusses the questions of real player's inclination to use IPD solution strategies in their pure form or to use the same strategy before and after experimental inter-ventions related to social identity formation. It is shown that experimentally induced socialization can be considered as a mechanism for increasing the degree of certainty in the choice of strategies when solving IPD task. It is found out that the models based on neural networks turn out to be more efficient after experi-mentally evoked social identity in a group of 6 people; and the models based on neural net-works are least effective in the case of predicting a subject's belonging to a gender group. When solving IPD problems by real people, it turns out to be possible to talk about generalized strategies that take into account not only the evolutionary properties of «pure» strategies, but also reflect various social factors.
Simulation of an Optional Strategy in the Prisoner's Dilemma in Spatial and Non-spatial Environments

Marcos Cardinot,Maud Gibbons,Colm O'Riordan,Josephine Griffith

DOI: https://doi.org/10.48550/arXiv.1608.05044

2016-08-17

Computer Science and Game Theory

Abstract:This paper presents research comparing the effects of different environments on the outcome of an extended Prisoner's Dilemma, in which agents have the option to abstain from playing the game. We consider three different pure strategies: cooperation, defection and abstinence. We adopt an evolutionary game theoretic approach and consider two different environments: the first which imposes no spatial constraints and the second in which agents are placed on a lattice grid. We analyse the performance of the three strategies as we vary the loner's payoff in both structured and unstructured environments. Furthermore we also present the results of simulations which identify scenarios in which cooperative clusters of agents emerge and persist in both environments.
An Adaptive Strategy Via Reinforcement Learning for the Prisoner's Dilemma Game

Lei Xue,Changyin Sun,Donald Wunsch,Yingjiang Zhou,Fang Yu

DOI: https://doi.org/10.1109/jas.2017.7510466

2018-01-01

IEEE/CAA Journal of Automatica Sinica

Abstract:The iterated prisoner’s dilemma（IPD） is an ideal model for analyzing interactions between agents in complex networks. It has attracted wide interest in the development of novel strategies since the success of tit-for-tat in Axelrod’s tournament. This paper studies a new adaptive strategy of IPD in different complex networks, where agents can learn and adapt their strategies through reinforcement learning method. A temporal difference learning method is applied for designing the adaptive strategy to optimize the decision making process of the agents. Previous studies indicated that mutual cooperation is hard to emerge in the IPD. Therefore, three examples which based on square lattice network and scale-free network are provided to show two features of the adaptive strategy. First, the mutual cooperation can be achieved by the group with adaptive agents under scale-free network, and once evolution has converged mutual cooperation, it is unlikely to shift. Secondly, the adaptive strategy can earn a better payoff compared with other strategies in the square network. The analytical properties are discussed for verifying evolutionary stability of the adaptive strategy.
Inferring Strategies from Observations in Long Iterated Prisoner's Dilemma Experiments

Eladio Montero-Porras,Jelena Grujic,Elias Fernandez-Domingos,Tom Lenaerts

DOI: https://doi.org/10.1038/s41598-022-11654-2

2022-02-09

Abstract:While many theoretical studies have revealed the strategies that could lead to and maintain cooperation in the Iterated Prisoner's Dilemma, less is known about what human participants actually do in this game and how strategies change when being confronted with anonymous partners in each round. Previous attempts used short experiments, made different assumptions of possible strategies, and led to very different conclusions. We present here two long treatments that differ in the partner matching strategy used, i.e. fixed or shuffled partners. Here we use unsupervised methods to cluster the players based on their actions and then Hidden Markov Model to infer what are those strategies in each cluster. Analysis of the inferred strategies reveals that fixed partner interaction leads to a behavioral self-organization. Shuffled partners generate subgroups of strategies that remain entangled, apparently blocking the self-selection process that leads to fully cooperating participants in the fixed partner treatment. Analyzing the latter in more detail shows that AllC, AllD, TFT- and WSLS-like behavior can be observed. This study also reveals that long treatments are needed as experiments less than 25 rounds capture mostly the learning phase participants go through in these kinds of experiments.

Computer Science and Game Theory
Friendly-rivalry solution to the iterated $n$-person public-goods game

Yohsuke Murase,Seung Ki Baek

DOI: https://doi.org/10.1371/journal.pcbi.1008217

2021-01-22

Abstract:Repeated interaction promotes cooperation among rational individuals under the shadow of future, but it is hard to maintain cooperation when a large number of error-prone individuals are involved. One way to construct a cooperative Nash equilibrium is to find a `friendly-rivalry' strategy, which aims at full cooperation but never allows the co-players to be better off. Recently it has been shown that for the iterated Prisoner's Dilemma in the presence of error, a friendly rival can be designed with the following five rules: Cooperate if everyone did, accept punishment for your own mistake, punish defection, recover cooperation if you find a chance, and defect in all the other circumstances. In this work, we construct such a friendly-rivalry strategy for the iterated $n$-person public-goods game by generalizing those five rules. The resulting strategy makes a decision with referring to the previous $m=2n-1$ rounds. A friendly-rivalry strategy for $n=2$ inherently has evolutionary robustness in the sense that no mutant strategy has higher fixation probability in this population than that of a neutral mutant. Our evolutionary simulation indeed shows excellent performance of the proposed strategy in a broad range of environmental conditions when $n= 2$ and $3$.

Physics and Society,Populations and Evolution
Iterated Prisoner’s Dilemma contains strategies that dominate any evolutionary opponent

William H. Press,Freeman J. Dyson

DOI: https://doi.org/10.1073/pnas.1206569109

IF: 11.1

2012-05-21

Proceedings of the National Academy of Sciences

Abstract:The two-player Iterated Prisoner’s Dilemma game is a model for both sentient and evolutionary behaviors, especially including the emergence of cooperation. It is generally assumed that there exists no simple ultimatum strategy whereby one player can enforce a unilateral claim to an unfair share of rewards. Here, we show that such strategies unexpectedly do exist. In particular, a player X who is witting of these strategies can ( i ) deterministically set her opponent Y’s score, independently of his strategy or response, or ( ii ) enforce an extortionate linear relation between her and his scores. Against such a player, an evolutionary player’s best response is to accede to the extortion. Only a player with a theory of mind about his opponent can do better, in which case Iterated Prisoner’s Dilemma is an Ultimatum Game.

multidisciplinary sciences
Research on Strategy Selection in N-Person Stochastic Iterated Game

ZHANG Si-hai,XU Min,WANG Xu-fa

DOI: https://doi.org/10.3969/j.issn.0253-2778.2006.11.008

2006-01-01

Journal of University of Science and Technology of China

Abstract:The strategy selection in 2-person iterated prisoner dilemma was extended and the concept of n-person stochastic iterated prisoner dilemma(NSIPD) was proposed to accommodate to new development arising from technology advancement.In NSIPD everyone's history information of choice could be seen by all players.At each game round,two players were selected randomly to play PD game.A computer tournament was used to search for the best strategy in NSIPD.The experimental results show that TIT-FOR-TAT and ALWAYS_DEFECT are not always the best strategy,and that the best strategy is closely correlated with repeated times and strategy distribution.
The robustness of zero-determinant strategies in Iterated Prisoner's Dilemma games.

Jing Chen,Aleksey Zinger

DOI: https://doi.org/10.1016/j.jtbi.2014.05.004

IF: 2.405

2014-01-01

Journal of Theoretical Biology

Abstract:Press and Dyson (2012) discovered a special set of strategies in two-player Iterated Prisoner׳s Dilemma games, the zero-determinant (ZD) strategies. Surprisingly, a player using such strategies can unilaterally enforce a linear relation between the payoffs of the two players. In particular, with a subclass of such strategies, the extortionate strategies, the former player obtains an advantageous share of the total payoff of the players, and the other player׳s best response is to always cooperate, by doing which he maximizes the payoff of the extortioner as well. When an extortionate player faces a player who is not aware of the theory of ZD strategies and improves his own payoff by adaptively changing his strategy following some unknown dynamics, Press and Dyson conjecture that there always exist adapting paths for the latter leading to the maximum possible scores for both players.
Zero-determinant strategies in iterated multi-strategy games

Jin-Li Guo

DOI: https://doi.org/10.48550/arXiv.1409.1786

2014-09-05

Computer Science and Game Theory

Abstract:Self-serving, rational agents sometimes cooperate to their mutual benefit. The two-player iterated prisoner's dilemma game is a model for including the emergence of cooperation. It is generally believed that there is no simple ultimatum strategy which a player can control the return of the other participants. The recent discovery of the powerful class of zero-determinant strategies in the iterated prisoner's dilemma dramatically expands our understanding of the classic game by uncovering strategies that provide a unilateral advantage to sentient players pitted against unwitting opponents. However, strategies in the prisoner's dilemma game are only two strategies. Are there these results for general multi-strategy games? To address this question, the paper develops a theory for zero-determinant strategies for multi-strategy games, with any number of strategies. The analytical results exhibit a similar yet different scenario to the case of two-strategy games. Zero-determinant strategies in iterated prisoner's dilemma can be seen as degenerate case of our results. The results are also applied to the snowdrift game, the hawk-dove game and the chicken game.
Strong and safe Nash equilibrium in some repeated 3-player games

Tadeusz Kufel,Sławomir Plaskacz,Joanna Zwierzchowska

DOI: https://doi.org/10.48550/arXiv.1711.10929

2018-05-15

Abstract:We consider a 3-player game in the normal form, in which each player has two actions. We assume that the game is symmetric and repeated infinitely many times. At each stage players make their choices knowing only the average payoffs from previous stages of all the players. A strategy of a player in the repeated game is a function defined on the convex hull of the set of payoffs. Our aim is to construct a strong Nash equilibrium in the repeated game, i.e. a strategy profile being resistant to deviations by coalitions. Constructed equilibrium strategies are safe, i.e. the non-deviating player payoff is not smaller than the equilibrium payoff in the stage game, and deviating players' payoffs do not exceed the non-deviating player payoff more than a positive constant which can be arbitrary small and chosen by the non-deviating player. Our construction is inspired by Smale's good strategies described in \cite{smale}, where the repeated Prisoner's Dilemma was considered. In proofs we use arguments based on approachability and strong approachability type results.

Optimization and Control
Win-stay-lose-learn Promotes Cooperation in the Spatial Prisoner's Dilemma Game.

Yongkui Liu,Xiaojie Chen,Lin Zhang,Long Wang,Matjaz Perc

DOI: https://doi.org/10.1371/journal.pone.0030689

IF: 3.7

2012-01-01

PLoS ONE

Abstract:Holding on to one's strategy is natural and common if the later warrants success and satisfaction. This goes against widespread simulation practices of evolutionary games, where players frequently consider changing their strategy even though their payoffs may be marginally different than those of the other players. Inspired by this observation, we introduce an aspiration-based win-stay-lose-learn strategy updating rule into the spatial prisoner's dilemma game. The rule is simple and intuitive, foreseeing strategy changes only by dissatisfied players, who then attempt to adopt the strategy of one of their nearest neighbors, while the strategies of satisfied players are not subject to change. We find that the proposed win-stay-lose-learn rule promotes the evolution of cooperation, and it does so very robustly and independently of the initial conditions. In fact, we show that even a minute initial fraction of cooperators may be sufficient to eventually secure a highly cooperative final state. In addition to extensive simulation results that support our conclusions, we also present results obtained by means of the pair approximation of the studied game. Our findings continue the success story of related win-stay strategy updating rules, and by doing so reveal new ways of resolving the prisoner's dilemma.
Beyond Pairwise Strategy Updating in the Prisoner's Dilemma Game

Xiaofeng Wang,Matjaz Perc,Yongkui Liu,Xiaojie Chen,Long Wang

DOI: https://doi.org/10.1038/srep00740

IF: 4.6

2012-01-01

Scientific Reports

Abstract:In spatial games players typically alter their strategy by imitating the most successful or one randomly selected neighbor. Since a single neighbor is taken as reference, the information stemming from other neighbors is neglected, which begets the consideration of alternative, possibly more realistic approaches. Here we show that strategy changes inspired not only by the performance of individual neighbors but rather by entire neighborhoods introduce a qualitatively different evolutionary dynamics that is able to support the stable existence of very small cooperative clusters. This leads to phase diagrams that differ significantly from those obtained by means of pairwise strategy updating. In particular, the survivability of cooperators is possible even by high temptations to defect and over a much wider uncertainty range. We support the simulation results by means of pair approximations and analysis of spatial patterns, which jointly highlight the importance of local information for the resolution of social dilemmas.
From extortion to generosity, the evolution of zero-determinant strategies in the prisoner's dilemma

Alexander J. Stewart,Joshua B. Plotkin

DOI: https://doi.org/10.48550/arXiv.1304.7205

2013-04-26

Populations and Evolution

Abstract:Recent work has revealed a new class of "zero-determinant" (ZD) strategies for iterated, two-player games. ZD strategies allow a player to unilaterally enforce a linear relationship between her score and her opponent's score, and thus achieve an unusual degree of control over both players' long-term payoffs. Although originally conceived in the context of classical, two-player game theory, ZD strategies also have consequences in evolving populations of players. Here we explore the evolutionary prospects for ZD strategies in the Iterated Prisoner's Dilemma (IPD). Several recent studies have focused on the evolution of "extortion strategies" - a subset of zero-determinant strategies - and found them to be unsuccessful in populations. Nevertheless, we identify a different subset of ZD strategies, called "generous ZD strategies", that forgive defecting opponents, but nonetheless dominate in evolving populations. For all but the smallest population sizes, generous ZD strategies are not only robust to being replaced by other strategies, but they also can selectively replace any non-cooperative ZD strategy. Generous strategies can be generalized beyond the space of ZD strategies, and they remain robust to invasion. When evolution occurs on the full set of all IPD strategies, selection disproportionately favors these generous strategies. In some regimes, generous strategies outperform even the most successful of the well-known Iterated Prisoner's Dilemma strategies, including win-stay-lose-shift.

Smale Strategies For The N-Person Iterated Prisoner's Dilemma

The Iterated Prisoner's Dilemma: Good Strategies and Their Dynamics

Properties of Winning Iterated Prisoner's Dilemma Strategies

Network-Iterated Prisoner's Dilemma

Invincible Strategies of Iterated Prisoner's Dilemma

An Investigation of N-person Prisoners' Dilemmas

Network Structures between Strategies in Iterated Prisoners' Dilemma Games

Press-Dyson Analysis of Asynchronous, Sequential Prisoner's Dilemma

Space Modeling of Problem Solving Strategies of “Prisoner's Dilemma”

Simulation of an Optional Strategy in the Prisoner's Dilemma in Spatial and Non-spatial Environments

An Adaptive Strategy Via Reinforcement Learning for the Prisoner's Dilemma Game

Inferring Strategies from Observations in Long Iterated Prisoner's Dilemma Experiments

Friendly-rivalry solution to the iterated $n$-person public-goods game

Iterated Prisoner’s Dilemma contains strategies that dominate any evolutionary opponent

Research on Strategy Selection in N-Person Stochastic Iterated Game

The robustness of zero-determinant strategies in Iterated Prisoner's Dilemma games.

Zero-determinant strategies in iterated multi-strategy games

Strong and safe Nash equilibrium in some repeated 3-player games

Win-stay-lose-learn Promotes Cooperation in the Spatial Prisoner's Dilemma Game.

Beyond Pairwise Strategy Updating in the Prisoner's Dilemma Game

From extortion to generosity, the evolution of zero-determinant strategies in the prisoner's dilemma