Abstract:Communication is an essential way for multi-agent system to coordinate. By sharing local observations and intentions via communication channel, agents can better deal with dynamic environment and thus make optimal decisions. However, restricted by the limited communication channel, agents have to leverage less communication resources to transmit more informative messages. In this article, we propose a two-level hierarchical multi-agent reinforcement learning algorithm which utilizes different timescales in different levels. Communication happens only between high levels at a coarser time scale to generate sub-goals which convey the intention of agents for the low level. And the low level is responsible for implementing these sub-goals by controlling primitive actions at every tick of environment. Sub-goal is the core of this hierachical communication architecture which requires the high level to communicate efficiently and provide guidance for the low level to coordinate. This hierarchical communication architecture conveys several benefits: 1) It coarsens the collaborative granularity and reduces the requirement of communication since communication happens only in high level at a larger scale; 2) It enables the high level to focus on the coordination of goals without paying attention to implementation, thus improves the efficiency of communication; and 3) It makes better control by dividing a complex multi-agent cooperative task into multiple single-agent tasks. In experiments, we apply our approach in vehicle collision avoidance tasks and achieve better performance than baselines.

Automatically Discovering Hierarchies in Multi-agent Reinforcement Learning

Automatic Option Generation in Hierarchical Reinforcement Learning Via Immune Clustering

Hierarchical Reinforcement Learning with an Automatically Generated Hierarchy Based on Immune Clustering

Option Automatic Generation in Hierarchical Reinforcement Learning

Moving Forward in Formation: A Decentralized Hierarchical Learning Approach to Multi-Agent Moving Together

Dynamic Hierarchies in Hierarchical Reinforcement Learning

Algorithm for Automatic Constructing Option Based on Multi-Agent

Parallel Automatic Hierarchy in Hierarchical Reinforcement Learning

Automatic Discovery and Transfer of Maxq Hierarchies in A Complex System

Subgoal-based Hierarchical Reinforcement Learning for Multi-Agent Collaboration

Multi-Agent Hierarchical Reinforcement Learning by Integrating Options into MAXQ

Deep Hierarchical Communication Graph in Multi-Agent Reinforcement Learning.

Causality-driven Hierarchical Structure Discovery for Reinforcement Learning

Hierarchical Multi-Agent Reinforcement Learning for Air Combat Maneuvering

Hierarchical State Extraction Method for Multi-Agent Environment

Hierarchical Reinforcement Learning for Multi-agent MOBA Game

Autonomic Discovery of Subgoals in Hierarchical Reinforcement Learning

Hierarchical Method for Cooperative Multiagent Reinforcement Learning in Markov Decision Processes

A Hierarchical Framework for Cooperative Tasks in Multi-agent Systems

An Evolutionary Approach for Optimizing Hierarchical Multi-Agent System Organization

Cooperative Multi-agent Reinforcement Learning with Hierachical Communication Architecture