Abstract:As a specific kind of cyber–physical systems (CPSs), autonomous robot clusters play an important role in various intelligent manufacturing fields. However, due to the increasing design complexity of robot clusters, it is becoming more and more challenging to guarantee the safety and efficiency for multirobot cooperative navigation in dynamic and complex environments. Although deep reinforcement learning (DRL) shows great potential in learning multirobot cooperative navigation policies, existing DRL-based approaches suffer from scalability issues and rarely consider the transferability of trained policies to new tasks. To address these problems, this article presents a novel DRL-based multirobot cooperative navigation approach named HRMR-Navi that equips each robot with both a two-layered hierarchical graph network model and an attention-based communication model. In our approach, the hierarchical graph network model can efficiently figure out hierarchical relations among all agents that either cooperate for efficiency or avoid obstacles for safety to derive more advanced strategies, and the communication model can accurately form a global view of the environment for a specific robot, thus, the multirobot cooperation efficiency can be further strengthened. Meanwhile, we propose an improved proximal policy optimization (PPO) algorithm based on the Maximum Entropy Reinforcement Learning, named MEPPO, to enhance the robot exploration ability. Comprehensive experimental results demonstrate that, compared with state-of-the-art approaches, HRMR-Navi can achieve more efficient cooperative navigation with less time cost, lower collision rate, higher scalability, and better knowledge transferability.

Hierarchical Representations and Explicit Memory: Learning Effective Navigation Policies on 3D Scene Graphs using Graph Neural Networks

Learning Hierarchical Graph-Based Policy for Goal-Reaching in Unknown Environments

Multi-Object Navigation with dynamically learned neural implicit representations

Learning Cross Dimension Scene Representation for Interactive Navigation Agents in Obstacle-Cluttered Environments

Learning Crowd Behaviors in Navigation with Attention-based Spatial-Temporal Graphs

DGMem: Learning Visual Navigation Policy Without Any Labels by Dynamic Graph Memory

Efficient Reinforcement Learning for 3D LiDAR Navigation of Mobile Robot

A Deep Learning Based Behavioral Approach to Indoor Autonomous Navigation

HSPNav: Hierarchical Scene Prior Learning for Visual Semantic Navigation Towards Real Settings

Task-Driven Graph Attention for Hierarchical Relational Object Navigation

3D-Aware Object Goal Navigation Via Simultaneous Exploration and Identification

Multi-robot Navigation with Graph Attention Neural Network and Hierarchical Motion Planning

Socially Aware Object Goal Navigation with Heterogeneous Scene Representation Learning

Learning with a Mole: Transferable latent spatial representations for navigation without reconstruction

Hierarchical Open-Vocabulary 3D Scene Graphs for Language-Grounded Robot Navigation

Sensor-Based Navigation Using Hierarchical Reinforcement Learning

MGRL: Graph neural network based inference in a Markov network with reinforcement learning for visual navigation

Robust Policies via Mid-Level Visual Representations: An Experimental Study in Manipulation and Navigation

Learning Navigational Visual Representations with Semantic Map Supervision

Learning Exploration Policies for Navigation

Hierarchical Relational Graph Learning for Autonomous Multirobot Cooperative Navigation in Dynamic Environments