Abstract:The Centralized Training and Decentralized Execution (CTDE) paradigm, where a centralized critic is allowed to access global information during the training phase while maintaining the learned policies executed with only local information in a decentralized way, has achieved great progress in recent years. Despite the progress, CTDE may suffer from the issue of Centralized-Decentralized Mismatch (CDM): the suboptimality of one agent's policy can exacerbate policy learning of other agents through the centralized joint critic. In contrast to centralized learning, the cooperative model that most closely resembles the way humans cooperate in nature is fully decentralized, i.e. Independent Learning (IL). However, there are still two issues that need to be addressed before agents coordinate through IL: (1) how agents are aware of the presence of other agents, and (2) how to coordinate with other agents to improve joint policy under IL. In this paper, we propose an inference-based coordinated MARL method: Deep Motor System (DMS). DMS first presents the idea of individual intention inference where agents are allowed to disentangle other agents from their environment. Secondly, causal inference was introduced to enhance coordination by reasoning each agent's effect on others' behavior. The proposed model was extensively experimented on a series of Multi-Agent MuJoCo and StarCraftII tasks. Results show that the proposed method outperforms independent learning algorithms and the coordination behavior among agents can be learned even without the CTDE paradigm compared to the state-of-the-art baselines including IPPO and HAPPO.

Coordination Theory-Based Model of Military Decision Making Process

Models of Strategic Decision-Making under Informational Control

Coordination and Delegation under Multitasks

AI-based Coordination Model of Organizational Decision Making

Model building of coordination theory: A review

Coordination Tactics and Team Performance:A Case Study Based on Scientific Reseach Teams

The Study on the Organizational Model of Coordination Association

A Model-Based Analysis of the Complexity of Collaborative Command-and-control System

Quantitative Analysis on Collaborative Efficiency of Large-Scale C2 Organization

Unifying warfighting functions in mathematical modelling: combat, manoeuvre, and C2

Research on the Coordination Mechanism among Modular Organizations

The Analysis on Team's Implicit Coordination Function and Its Influence Factors

Cognitive Modeling for Crew's Information Processing Based on Multi-resource Theory

Coordination as inference in multi-agent reinforcement learning

Research on Coordination Mechanism Model of Launch Vehicle Supply Chain Based on Complex System Theory

Coalition formation problem: a capability-centric analysis and general model

Interactions between dynamic team composition and coordination: An agent-based modeling approach

Assessment method for effectiveness of combat command based on information entropy

The cost of coordination can exceed the benefit of collaboration in performing complex tasks

A Cooperative Effect-Based Decision Support Model for Team Formation.

Multiple Attribute Decision Making Approach to Attacking Order of Battlefields Targets