Abstract:Existing value-factorized based Multi-Agent deep Reinforce-ment Learning (MARL) approaches are well-performing invarious multi-agent cooperative environment under thecen-tralized training and decentralized execution(CTDE) scheme,where all agents are trained together by the centralized valuenetwork and each agent execute its policy independently. How-ever, an issue remains open: in the centralized training process,when the environment for the team is partially observable ornon-stationary, i.e., the observation and action informationof all the agents cannot represent the global states, existingmethods perform poorly and sample inefficiently. Regret Min-imization (RM) can be a promising approach as it performswell in partially observable and fully competitive <a class="link-external link-http" href="http://settings.However" rel="external noopener nofollow">this http URL</a>, it tends to model others as opponents and thus can-not work well under the CTDE scheme. In this work, wepropose a novel team RM based Bayesian MARL with threekey contributions: (a) we design a novel RM method to traincooperative agents as a team and obtain a team regret-basedpolicy for that team; (b) we introduce a novel method to de-compose the team regret to generate the policy for each agentfor decentralized execution; (c) to further improve the perfor-mance, we leverage a differential particle filter (a SequentialMonte Carlo method) network to get an accurate estimation ofthe state for each agent. Experimental results on two-step ma-trix games (cooperative game) and battle games (large-scalemixed cooperative-competitive games) demonstrate that ouralgorithm significantly outperforms state-of-the-art methods.

Qrelation: an Agent Relation-Based Approach for Multi-Agent Reinforcement Learning Value Function Factorization.

QFree: A Universal Value Function Factorization for Multi-Agent Reinforcement Learning

S2rl

Is Centralized Training with Decentralized Execution Framework Centralized Enough for MARL?

Multiagent Q-learning with Sub-Team Coordination.

Learning Multi-Agent Cooperation via Considering Actions of Teammates

ConcaveQ: Non-Monotonic Value Function Factorization Via Concave Representations in Deep Multi-Agent Reinforcement Learning

QPLEX: Duplex Dueling Multi-Agent Q-Learning.

Q-value Path Decomposition for Deep Multiagent Reinforcement Learning

Towards Understanding Cooperative Multi-Agent Q-Learning with Value Factorization.

Multi-Agent Collaboration via Reward Attribution Decomposition

DVF:Multi-agent Q-learning with difference value factorization

Credit-of-Q-value for Multi-Agent Reinforcement Learning

Attentive Relational State Representation in Decentralized Multiagent Reinforcement Learning.

Multi-agent Dueling Q-learning with Mean Field and Value Decomposition

More Centralized Training, Still Decentralized Execution: Multi-Agent Conditional Policy Factorization

Attentional Factorized Q-Learning for Many-Agent Learning.

CTDS: Centralized Teacher with Decentralized Student for Multi-Agent Reinforcement Learning

Hierarchical relationship modeling in multi-agent reinforcement learning for mixed cooperative–competitive environments

Value Functions Factorization with Latent State Information Sharing in Decentralized Multi-Agent Policy Gradients

Inducing Cooperation via Team Regret Minimization based Multi-Agent Deep Reinforcement Learning