Abstract:The autonomous exploration of environments by multi-robot systems is a critical task with broad applications in rescue missions, exploration endeavors, and beyond. Current approaches often rely on either greedy frontier selection or end-to-end deep reinforcement learning (DRL) methods, yet these methods are frequently hampered by limitations such as short-sightedness, overlooking long-term implications, and convergence difficulties stemming from the intricate high-dimensional learning space. To address these challenges, this paper introduces an innovative integration strategy that combines the low-dimensional action space efficiency of frontier-based methods with the far-sightedness and optimality of DRL-based approaches. We propose a three-tiered planning framework that first identifies frontiers in free space, creating a sparse map representation that lightens data transmission burdens and reduces the DRL action space's dimensionality. Subsequently, we develop a multi-graph neural network (mGNN) that incorporates states of potential targets and robots, leveraging policy-based reinforcement learning to compute affinities, thereby superseding traditional heuristic utility values. Lastly, we implement local routing planning through subsequence search, which avoids exhaustive sequence traversal. Extensive validation across diverse scenarios and comprehensive simulation results demonstrate the effectiveness of our proposed method. Compared to baseline approaches, our framework achieves environmental exploration with fewer time steps and a notable reduction of over 30% in data transmission, showcasing its superiority in terms of efficiency and performance.

RTP-Q: a Reinforcement Learning System with an Active Exploration Planning Structure for Enhancing the Convergence Rate

Rtp-Q: A Reinforcement Learning System with Time Constraints Exploration Planning for Accelerating the Learning Rate

Efficient Autonomous Exploration of Unknown Environment Using Regions Segmentation and VRP

A Reinforcement Learning Based RRT Algorithm with Value Estimation

Deep Reinforcement Learning Integrated RRT Algorithm for Path Planning

RBI-RRT*: Efficient Sampling-based Path Planning for High-dimensional State Space

DSVP: Dual-Stage Viewpoint Planner for Rapid Exploration by Dynamic Expansion

An Enhanced Hierarchical Planning Framework for Multi-Robot Autonomous Exploration

Generative Planning for Temporally Coordinated Exploration in Reinforcement Learning

An Incremental Optimization Approach to Address the Spatiotemporal Reward Coupling Effects in Deep Reinforcement Learning for Path Planning

Deep Reinforcement Learning-based Large-scale Robot Exploration

Hierarchical path planner for unknown space exploration using reinforcement learning-based intelligent frontier selection

Planning with Exploration: Addressing Dynamics Bottleneck in Model-based Reinforcement Learning

Explicit Planning for Efficient Exploration in Reinforcement Learning.

Hybrid Bidirectional Rapidly Exploring Random Tree Path Planning Algorithm with Reinforcement Learning

Reinforcement Learning with Probabilistically Complete Exploration

Application research of RRT algorithm path planning based on reinforcement learning

Learning to Plan in High Dimensions via Neural Exploration-Exploitation Trees

Improve RRT Algorithm for Path Planning in Complex Environments

Efficient and Scalable Exploration Via Estimation-Error

A Hierarchical SLAM Framework Based on Deep Reinforcement Learning for Active Exploration