Abstract:The Centralized Training and Decentralized Execution (CTDE) paradigm, where a centralized critic is allowed to access global information during the training phase while maintaining the learned policies executed with only local information in a decentralized way, has achieved great progress in recent years. Despite the progress, CTDE may suffer from the issue of Centralized-Decentralized Mismatch (CDM): the suboptimality of one agent's policy can exacerbate policy learning of other agents through the centralized joint critic. In contrast to centralized learning, the cooperative model that most closely resembles the way humans cooperate in nature is fully decentralized, i.e. Independent Learning (IL). However, there are still two issues that need to be addressed before agents coordinate through IL: (1) how agents are aware of the presence of other agents, and (2) how to coordinate with other agents to improve joint policy under IL. In this paper, we propose an inference-based coordinated MARL method: Deep Motor System (DMS). DMS first presents the idea of individual intention inference where agents are allowed to disentangle other agents from their environment. Secondly, causal inference was introduced to enhance coordination by reasoning each agent's effect on others' behavior. The proposed model was extensively experimented on a series of Multi-Agent MuJoCo and StarCraftII tasks. Results show that the proposed method outperforms independent learning algorithms and the coordination behavior among agents can be learned even without the CTDE paradigm compared to the state-of-the-art baselines including IPPO and HAPPO.

Multi-agent coordination strategy estimation method based on control domain

Local Multi-Robot Coordination and Experiments

A Multi-agent Formation Control Algorithm in Interference Environment

Distributed Formation Control with Open-Loop Nash Strategy

Mechanism and Strategy of Multi-robot Coordination for Exploring Unknown Environments

Cooperative reference frame estimation for multi-agent systems via formation control

A Distributed Multi-Robot Cooperation Strategy Classification

A new coordination method for multi-robot system

A Plan Recognition Approach for Agent in Adversarial Multi-Agent System

On the coordination efficiency of strategic multi-agent robotic teams

Coordination as inference in multi-agent reinforcement learning

A muti-agent defensive strategy based on monte carlo method

Cooperative-competitive Game Based Approach to the Local Path Planning Problem of Distributed Multi-Agent Systems

Current status and development of multi-robot coordinative and cooperative systems

Multi-Agent Vision Positioning and Coordination Formation Control Strategy of Agricultural Machinery

A Cooperative Multi-Agent Reinforcement Learning Method Based on Coordination Degree

Cooperative Control Design for Multi-agent System Inspection on Power Substations

An Auction-based Coordination Strategy for Task-Constrained Multi-Agent Stochastic Planning with Submodular Rewards

Formation constrained multi-agent control

Stable Distributed Model Predictive Control Strategy Based on Agent Coordination

A Novel Strategy for Distributed Multi-robot Coordination in Area Exploration