Abstract:Using simulations between pairs of $\epsilon$-greedy q-learners with one-period memory, this article demonstrates that the potential function of the stochastic replicator dynamics (Foster and Young, 1990) allows it to predict the emergence of error-proof cooperative strategies from the underlying parameters of the repeated prisoner's dilemma. The observed cooperation rates between q-learners are related to the ratio between the kinetic energy exerted by the polar attractors of the replicator dynamics under the grim trigger strategy. The frontier separating the parameter space conducive to cooperation from the parameter space dominated by defection can be found by setting the kinetic energy ratio equal to a critical value, which is a function of the discount factor, $f(\delta) = \delta/(1-\delta)$, multiplied by a correction term to account for the effect of the algorithms' exploration probability. The gradient at the frontier increases with the distance between the game parameters and the hyperplane that characterizes the incentive compatibility constraint for cooperation under grim trigger. Building on literature from the neurosciences, which suggests that reinforcement learning is useful to understanding human behavior in risky environments, the article further explores the extent to which the frontier derived for q-learners also explains the emergence of cooperation between humans. Using metadata from laboratory experiments that analyze human choices in the infinitely repeated prisoner's dilemma, the cooperation rates between humans are compared to those observed between q-learners under similar conditions. The correlation coefficients between the cooperation rates observed for humans and those observed for q-learners are consistently above $0.8$. The frontier derived from the simulations between q-learners is also found to predict the emergence of cooperation between humans.

Learning Reciprocity in Complex Sequential Social Dilemmas

Multi-agent Reinforcement Learning in Sequential Social Dilemmas

Reciprocal Social Strategy in Social Repeated Games.

Reciprocal Reward Influence Encourages Cooperation From Self-Interested Agents

Inequity aversion improves cooperation in intertemporal social dilemmas

Reciprocal Social Strategy in Social Repeated Games and Emergence of Social Norms.

Preferential Learning and Memory Resolve Social Dilemma

Birds of a Feather Flock Together: A Close Look at Cooperation Emergence via Multi-Agent RL

The Evolution of Cooperation in Spatial Multigame with Voluntary Participation

Achieving cooperation through deep multiagent reinforcement learning in sequential prisoner's dilemmas

Effects of Reciprocal Rewarding on the Evolution of Cooperation in Voluntary Social Dilemmas

Towards Cooperation in Sequential Prisoner's Dilemmas: a Deep Multiagent Reinforcement Learning Approach

Admission-Based Reinforcement-Learning Algorithm in Sequential Social Dilemmas

Intrinsic fluctuations of reinforcement learning promote cooperation

Learning Fair Cooperation in Mixed-Motive Games with Indirect Reciprocity

A theoretical analysis of temporal difference learning in the iterated prisoner's dilemma game

Reciprocal Altruism-based Cooperation in a Social Network Game

Cooperation and Control in Multiplayer Social Dilemmas

Learning dynamics explains human behavior in Prisoner's Dilemma on networks

On the Emergence of Cooperation in the Repeated Prisoner's Dilemma

Using deep reinforcement learning to promote sustainable human behaviour on a common pool resource problem