Abstract:Cooperation as a self-organized collective behavior plays a significant role in the evolution of ecosystems and human society. Reinforcement learning (RL) offers a new perspective, distinct from imitation learning in evolutionary games, for exploring the mechanisms underlying its emergence. However, most existing studies with the public good game (PGG) employ a self-regarding setup or are on pairwise interaction networks. Players in the real world, however, optimize their policies based not only on their histories but also on the histories of their co-players, and the game is played in a group manner. In the work, we investigate the evolution of cooperation in the PGG under the other-regarding reinforcement learning evolutionary game (OR-RLEG) on hypergraph by combining the Q-learning algorithm and evolutionary game framework, where other players' action history is incorporated and the game is played on hypergraphs. Our results show that as the synergy factor increases, the parameter interval is divided into three distinct regions, the absence of cooperation (AC), medium cooperation (MC), and high cooperation (HC), accompanied by two abrupt transitions in the cooperation level near two transition points, respectively. Interestingly, we identify regular and anti-coordinated chessboard structures in the spatial pattern that positively contribute to the first cooperation transition but adversely affect the second. Furthermore, we provide a theoretical treatment for the first transition with an approximated first transition point and reveal that players with a long-sighted perspective and low exploration rate are more likely to reciprocate kindness with each other, thus facilitating the emergence of cooperation. Our findings contribute to understanding the evolution of human cooperation, where other-regarding information and group interactions are commonplace.

Emergence of anti-coordinated patterns in snowdrift game by reinforcement learning

Emergence of cooperation in two-agent repeated games with reinforcement learning

Universal Effect of Dynamical Reinforcement Learning Mechanism in Spatial Evolutionary Games

Oscillatory evolution of collective behavior in evolutionary games played with reinforcement learning

Evolution of Cooperation in Snowdrift Game with Multi-Memory Mechanism

Evolutionary Snowdrift Game with an Additional Strategy in Fully Connected Networks and Regular Lattices

Spatial Structure Favors Cooperative Behavior in the Snowdrift Game with Multiple Interactive Dynamics

Evolutionary Game Dynamics of Multi-Agent Cooperation Driven by Self-Learning

Cooperative Dynamics of Snowdrift Game on Spatial Distance-Dependent Small-World Networks

Emergence of Cooperation Induced by Preferential Learning

Evolutionary Snowdrift Game with Loners

Self organizing optimization and phase transition in reinforcement learning minority game system

Exploring Dominant Strategies in Iterated and Evolutionary Games: a Multi-Agent Reinforcement Learning Approach

Birds of a Feather Flock Together: A Close Look at Cooperation Emergence via Multi-Agent RL

Evolutionary games and spatial periodicity

Memory-Based Snowdrift Game On Networks

Collective Emergence in A Multi-Agent Network with Adaptive Games

Cooperation in Public Goods Games: Leveraging Other-Regarding Reinforcement Learning on Hypergraphs

Neural Auto-Curricula in Two-Player Zero-Sum Games.

Spatial structure often inhibits the evolution of cooperation in the snowdrift game