Abstract:This paper investigates the multi-agent cooperative exploration problem, which requires multiple agents to explore an unseen environment via sensory signals in a limited time. A popular approach to exploration tasks is to combine active mapping with planning. Metric maps capture the details of the spatial representation, but are with high communication traffic and may vary significantly between scenarios, resulting in inferior generalization. Topological maps are a promising alternative as they consist only of nodes and edges with abstract but essential information and are less influenced by the scene structures. However, most existing topology-based exploration tasks utilize classical methods for planning, which are time-consuming and sub-optimal due to their handcrafted design. Deep reinforcement learning (DRL) has shown great potential for learning (near) optimal policies through fast end-to-end inference. In this paper, we propose Multi-Agent Neural Topological Mapping (MANTM) to improve exploration efficiency and generalization for multi-agent exploration tasks. MANTM mainly comprises a Topological Mapper and a novel RL-based Hierarchical Topological Planner (HTP). The Topological Mapper employs a visual encoder and distance-based heuristics to construct a graph containing main nodes and their corresponding ghost nodes. The HTP leverages graph neural networks to capture correlations between agents and graph nodes in a coarse-to-fine manner for effective global goal selection. Extensive experiments conducted in a physically-realistic simulator, Habitat, demonstrate that MANTM reduces the steps by at least 26.40% over planning-based baselines and by at least 7.63% over RL-based competitors in unseen scenarios.

TOMA: Topological Map Abstraction for Reinforcement Learning

Active Neural Topological Mapping for Multi-Agent Exploration

Graph learning-based generation of abstractions for reinforcement learning

TOM: Learning Policy-Aware Models for Model-Based Reinforcement Learning via Transition Occupancy Matching

Learning to Navigate in Mazes with Novel Layouts using Abstract Top-down Maps

TopoNav: Topological Navigation for Efficient Exploration in Sparse Reward Environments

MGRL: Graph neural network based inference in a Markov network with reinforcement learning for visual navigation

DVT-Tree: Dynamic Visible Topology Tree for Efficient Mapless Navigation in Maze Environments

Leveraging Topological Maps in Deep Reinforcement Learning for Multi-Object Navigation

Explore with Dynamic Map: Graph Structured Reinforcement Learning

Graph Convolution with Topology Refinement for Automatic Reinforcement Learning

TopV-Nav: Unlocking the Top-View Spatial Reasoning Potential of MLLM for Zero-shot Object Navigation

EMExplorer: an Episodic Memory Enhanced Autonomous Exploration Strategy with Voronoi Domain Conversion and Invalid Action Masking

Learning to plan with uncertain topological maps

Learning Cognitive Maps from Transformer Representations for Efficient Planning in Partially Observed Environments

Topological Experience Replay

DRL-Tomo: a deep reinforcement learning-based approach to augmented data generation for network tomography

MR-TopoMap: Multi-Robot Exploration Based on Topological Map in Communication Restricted Environment.

Efficient Exploration and Discriminative World Model Learning with an Object-Centric Abstraction

Hierarchical Representations and Explicit Memory: Learning Effective Navigation Policies on 3D Scene Graphs using Graph Neural Networks

A transformer-based deep reinforcement learning approach to spatial navigation in a partially observable Morris Water Maze