Abstract:Gradient-based method has been extensively used in today's multiagent reinforcement learning (MARL). In a gradient-based MARL algorithm, each agent updates its parameterized strategy in the direction of the gradient of some performance index. However, studies on the convergence of the existing gradient-based MARL algorithms for identical interest games are quite few. In this article, we propose a policy gradient potential (PGP) algorithm that takes PGP as the source of information for guiding the strategy update, as opposed to the gradient itself, to learn the optimal joint strategy that has a maximal global reward. Since the payoff matrix and the joint strategy are often unavailable to the learning agents in reality, we consider the probability of obtaining the maximal reward as the performance index. Theoretical analysis of the PGP algorithm on the continuous model involving an identical interest repeated game shows that if the component action of every optimal joint action is unique, the critical points corresponding to all optimal joint actions are asymptotically stable. The PGP algorithm is experimentally studied and compared against other MARL algorithms on two commonly used collaborative tasks—the robots leaving a room task and the distributed sensor network task, as well as a real-world minefield navigation problem where only local state and local reward information are available. The results show that the PGP algorithm outperforms the other algorithms in terms of the cumulative reward and the number of time steps used in an episode.

An Empirical Study on Google Research Football Multi-agent Scenarios

Boosting Studies of Multi-Agent Reinforcement Learning on Google Research Football Environment: the Past, Present, and Future

Learning to Cooperate: Application of Deep Reinforcement Learning for Online AGV Path Finding.

Google Research Football: A Novel Reinforcement Learning Environment

From mimic to counteract: a two-stage reinforcement learning algorithm for Google research football

TiKick: Toward Playing Multi-agent Football Full Games from Single-agent Demonstrations

TiKick: Towards Playing Multi-agent Football Full Games from Single-agent Demonstrations

Embedding Contextual Information through Reward Shaping in Multi-Agent Learning: A Case Study from Google Football

Containerized Distributed Value-Based Multi-Agent Reinforcement Learning

MARLadona -- Towards Cooperative Team Play Using Multi-Agent Reinforcement Learning

Graph Neural Network based Agent in Google Research Football

Revisiting Some Common Practices in Cooperative Multi-Agent Reinforcement Learning

Diversity is Strength: Mastering Football Full Game with Interactive Reinforcement Learning of Multiple AIs

Heterogeneous Multi-Agent Reinforcement Learning for Zero-Shot Scalable Collaboration

A Meta-Game Evaluation Framework for Deep Multiagent Reinforcement Learning

FoX: Formation-aware exploration in multi-agent reinforcement learning

MARL-LNS: Cooperative Multi-agent Reinforcement Learning via Large Neighborhoods Search

Off-Agent Trust Region Policy Optimization

A Collaborative Multiagent Reinforcement Learning Method Based on Policy Gradient Potential

An Overview of Multi-Agent Reinforcement Learning from Game Theoretical Perspective

Arena: A General Evaluation Platform and Building Toolkit for Multi-Agent Intelligence