Abstract:Existing value-factorized based Multi-Agent deep Reinforce-ment Learning (MARL) approaches are well-performing invarious multi-agent cooperative environment under thecen-tralized training and decentralized execution(CTDE) scheme,where all agents are trained together by the centralized valuenetwork and each agent execute its policy independently. How-ever, an issue remains open: in the centralized training process,when the environment for the team is partially observable ornon-stationary, i.e., the observation and action informationof all the agents cannot represent the global states, existingmethods perform poorly and sample inefficiently. Regret Min-imization (RM) can be a promising approach as it performswell in partially observable and fully competitive <a class="link-external link-http" href="http://settings.However" rel="external noopener nofollow">this http URL</a>, it tends to model others as opponents and thus can-not work well under the CTDE scheme. In this work, wepropose a novel team RM based Bayesian MARL with threekey contributions: (a) we design a novel RM method to traincooperative agents as a team and obtain a team regret-basedpolicy for that team; (b) we introduce a novel method to de-compose the team regret to generate the policy for each agentfor decentralized execution; (c) to further improve the perfor-mance, we leverage a differential particle filter (a SequentialMonte Carlo method) network to get an accurate estimation ofthe state for each agent. Experimental results on two-step ma-trix games (cooperative game) and battle games (large-scalemixed cooperative-competitive games) demonstrate that ouralgorithm significantly outperforms state-of-the-art methods.

Accumulating Risk Capital Through Investing in Cooperation

Expert demonstrations guide reward decomposition for multi-agent cooperation

Cooperation with dynamic asymmetric evaluation in complex networks from a risk perspective

Promoting Cooperation in the Public Goods Game using Artificial Intelligent Agents

Learning Collective Action under Risk Diversity

A Game-Theoretic Framework for Managing Risk in Multi-Agent Systems

Intrinsic fluctuations of reinforcement learning promote cooperation

Risk aversion can promote cooperation

Emergent Cooperation under Uncertain Incentive Alignment

Inducing Cooperation via Team Regret Minimization based Multi-Agent Deep Reinforcement Learning

On the Complexity of Learning to Cooperate with Populations of Socially Rational Agents

Aligning Individual and Collective Objectives in Multi-Agent Cooperation

Escaping the State of Nature: A Hobbesian Approach to Cooperation in Multi-agent Reinforcement Learning

Cooperation in a non-Ergodic World on a Network -- Insurance and Beyond

Toward Risk-based Optimistic Exploration for Cooperative Multi-Agent Reinforcement Learning

Solving the collective-risk social dilemma with risky assets in well-mixed and structured populations

Robust Multi-Agent Reinforcement Learning Driven by Correlated Equilibrium

Improved cooperation by balancing exploration and exploitation in intertemporal social dilemma tasks

Birds of a Feather Flock Together: A Close Look at Cooperation Emergence via Multi-Agent RL

Cooperation in a non-ergodic world on a network - insurance and beyond