Abstract:Many real-world cooperative problems can be implemented using Multi-Agent Reinforcement Learning (MARL) techniques, such as urban traffic control or multi-role games. However, the policy learning of MARL algorithms contains features of long-trajectory training and partial observability, which leads to the sparsity of reward and the lack of decision information. To solve the above issues, this article studies hierarchical deep MARL and proposes a novel model named Hierarchical Spatio-Temporal Communication Network (HSTCN). HSTCN designs hierarchical policies with two-time granularities: high-level and low-level policies. All agents are jointly entered into a joint policy containing the above two policies, and each has its execution policy. Specifically, the high-level policy provides intrinsic goals and continuous reward samples for the low-level policy to alleviate reward sparsity. The Low-level policy absorbs the above information to improve the efficiency of the agents' execution policies and interact with the environment to optimize the next reward. What's more, the high-level policy designs a graph-like structural model with Spatio-Temporal abstract. The Spatio-Temporal model expands receptive fields to receive neighborhood information and facilitates learning more robust policies by capturing the underlying graph's spatial dependencies and temporal dynamics. Meanwhile, an evaluation network is added to increase the robustness. Empirically, we demonstrated the effectiveness of HSTCN in a long-trajectory training environment through Simulation of Urban MObility (SUMO), while StarCraft II maps are tested as abstract environment. The above experimental results prove that the performance of HSTCN is superior to other advanced algorithms and verify the rationality of HSTCN design.

Multi-mobile Object Motion Coordination with Reinforcement Learning.

A Novel Motion Coordination Method for Variable-Sized Multi-Mobile Robots

Motion Coordination Of Multiple Robots Based On Deep Reinforcement Learning

Collision-aware Multi-robot Motion Coordination Deep-RL with Dynamic Priority Strategy

Reinforcement Learning Compensated Coordination Control of Multiple Mobile Manipulators for Tight Cooperation

Deep reinforcement learning for multi-objective combinatorial optimization: A case study on multi-objective traveling salesman problem

Hierarchical and Stable Multiagent Reinforcement Learning for Cooperative Navigation Control

Multi-robot Cooperation Based on Hierarchical Reinforcement Learning

Reinforcement Learning Control for Coordinated Manipulation of Multi-Robots.

Learning to Coordinate for a Worker-Station Multi-robot System in Planar Coverage Tasks

Behavioral Control Task Supervisor with Memory Based on Reinforcement Learning for Human-Multi-robot Coordination Systems

Coordinated Multiagent Reinforcement Learning for Teams of Mobile Sensing Robots.

Multi-robot Cooperative Object Transportation using Decentralized Deep Reinforcement Learning

Multi-Agent Coordination Method Based On Fuzzy Q-Learning

A Model-Based Solution to the Offline Multi-Agent Reinforcement Learning Coordination Problem

Coordination as inference in multi-agent reinforcement learning

Coordination Learning Control for Groups of Mobile Agents.

A Reinforcement Learning Mission Supervisor with Memory for Human-multi-robot Coordination Systems

Hierarchical Coordination Multi-Agent Reinforcement Learning with Spatio-Temporal Abstraction

Coordinated Control Based on Reinforcement Learning for Dual-Arm Continuum Manipulators in Space Capture Missions