Abstract:The Centralized Training and Decentralized Execution (CTDE) paradigm, where a centralized critic is allowed to access global information during the training phase while maintaining the learned policies executed with only local information in a decentralized way, has achieved great progress in recent years. Despite the progress, CTDE may suffer from the issue of Centralized-Decentralized Mismatch (CDM): the suboptimality of one agent's policy can exacerbate policy learning of other agents through the centralized joint critic. In contrast to centralized learning, the cooperative model that most closely resembles the way humans cooperate in nature is fully decentralized, i.e. Independent Learning (IL). However, there are still two issues that need to be addressed before agents coordinate through IL: (1) how agents are aware of the presence of other agents, and (2) how to coordinate with other agents to improve joint policy under IL. In this paper, we propose an inference-based coordinated MARL method: Deep Motor System (DMS). DMS first presents the idea of individual intention inference where agents are allowed to disentangle other agents from their environment. Secondly, causal inference was introduced to enhance coordination by reasoning each agent's effect on others' behavior. The proposed model was extensively experimented on a series of Multi-Agent MuJoCo and StarCraftII tasks. Results show that the proposed method outperforms independent learning algorithms and the coordination behavior among agents can be learned even without the CTDE paradigm compared to the state-of-the-art baselines including IPPO and HAPPO.

Coordination-Policy Based Composed System Behavior Derivation

Synthesis of logical controllers and discrete-event systems supervisory control theory

Deriving Available Behavior All out from Incompatible Component Compositions

Ontology-Based Conceptual Modeling Of Policy-Driven Control Framework: Oriented To Multi-Agent System For Web Services Management

Behavioral Compatibility Analysis and Optimal Control Policy in Web Services Composition

Scenario-Based Component Behavior Derivation.

Pure Coordination using the Coordinator--Configurator Pattern

An Approach for Harmonizing Conflicting Policies in Multiple Self-Adaptive Modules

Cooperative Problem Solving Process Based on MAS_NP Model

PD-Agent: A Flexible Agent BDI Model for Autonomic Computing

Policy-Driven Agent Social

Correct by Design Coordination of Autonomous Driving Systems

Agent behaviors and coordinative mechanism

An Approach to Composing Multiple Component Implementations for Satisfying Quality Requirements

Learning-based Formal Synthesis of Cooperative Multi-agent Systems

A Tool to Construct One Comprehensive Legal Environment for Behavioral Incompatible Components

A values-driven self-organization mechanism for automating multiagent coordination.

Coordination as inference in multi-agent reinforcement learning

An Extended Agent BDI Model with Norms, Policies and Contracts

On attribute-based usage control policy ratification for cooperative computing context

Safety-Constrained Policy Transfer with Successor Features