Abstract:Existing value-factorized based Multi-Agent deep Reinforce-ment Learning (MARL) approaches are well-performing invarious multi-agent cooperative environment under thecen-tralized training and decentralized execution(CTDE) scheme,where all agents are trained together by the centralized valuenetwork and each agent execute its policy independently. How-ever, an issue remains open: in the centralized training process,when the environment for the team is partially observable ornon-stationary, i.e., the observation and action informationof all the agents cannot represent the global states, existingmethods perform poorly and sample inefficiently. Regret Min-imization (RM) can be a promising approach as it performswell in partially observable and fully competitive <a class="link-external link-http" href="http://settings.However" rel="external noopener nofollow">this http URL</a>, it tends to model others as opponents and thus can-not work well under the CTDE scheme. In this work, wepropose a novel team RM based Bayesian MARL with threekey contributions: (a) we design a novel RM method to traincooperative agents as a team and obtain a team regret-basedpolicy for that team; (b) we introduce a novel method to de-compose the team regret to generate the policy for each agentfor decentralized execution; (c) to further improve the perfor-mance, we leverage a differential particle filter (a SequentialMonte Carlo method) network to get an accurate estimation ofthe state for each agent. Experimental results on two-step ma-trix games (cooperative game) and battle games (large-scalemixed cooperative-competitive games) demonstrate that ouralgorithm significantly outperforms state-of-the-art methods.

Competitive Multi-agent Deep Reinforcement Learning with Counterfactual Thinking

Counterfactual-Based Action Evaluation Algorithm in Multi-Agent Reinforcement Learning

Optimal Exploration Algorithm of Multi-Agent Reinforcement Learning Methods (Student Abstract)

Dueling Network Architecture for Multi-Agent Deep Deterministic Policy Gradient

An Improved Approach Towards Multi-Agent Pursuit–Evasion Game Decision-Making Using Deep Reinforcement Learning

A Dynamically Adaptive Approach to Reducing Strategic Interference for Multi-agent Systems

Toward a Psychology of Deep Reinforcement Learning Agents Using a Cognitive Architecture

Counter-Factual Reinforcement Learning: How to Model Decision-Makers That Anticipate The Future

From mimic to counteract: a two-stage reinforcement learning algorithm for Google research football

Inducing Cooperation via Team Regret Minimization based Multi-Agent Deep Reinforcement Learning

Theory of Mind as Intrinsic Motivation for Multi-Agent Reinforcement Learning

Episodic Future Thinking Mechanism for Multi-agent Reinforcement Learning

Decentralized Counterfactual Value with Threat Detection for Multi-Agent Reinforcement Learning in Mixed Cooperative and Competitive Environments

Algorithms in Multi-Agent Systems: A Holistic Perspective from Reinforcement Learning and Game Theory

A new multi-agent reinforcement learning approach

Modeling Theory of Mind in Multi-Agent Games Using Adaptive Feedback Control

Pangu-Agent: A Fine-Tunable Generalist Agent with Structured Reasoning

Hindsight-aware Deep Reinforcement Learning Algorithm for Multi-Agent Systems

Improving Agent Decision Payoffs via a New Framework of Opponent Modeling

Cognition-Oriented Multiagent Reinforcement Learning

Counterfactual State Explanations for Reinforcement Learning Agents via Generative Deep Learning