Abstract:Effective exploration is crucial to discovering optimal strategies for multi-agent reinforcement learning (MARL) in complex coordination tasks. Existing methods mainly utilize intrinsic rewards to enable committed exploration or use role-based learning for decomposing joint action spaces instead of directly conducting a collective search in the entire action-observation space. However, they often face challenges obtaining specific joint action sequences to reach successful states in long-horizon tasks. To address this limitation, we propose Imagine, Initialize, and Explore (IIE), a novel method that offers a promising solution for efficient multi-agent exploration in complex scenarios. IIE employs a transformer model to imagine how the agents reach a critical state that can influence each other's transition functions. Then, we initialize the environment at this state using a simulator before the exploration phase. We formulate the imagination as a sequence modeling problem, where the states, observations, prompts, actions, and rewards are predicted autoregressively. The prompt consists of timestep-to-go, return-to-go, influence value, and one-shot demonstration, specifying the desired state and trajectory as well as guiding the action generation. By initializing agents at the critical states, IIE significantly increases the likelihood of discovering potentially important under-explored regions. Despite its simplicity, empirical results demonstrate that our method outperforms multi-agent exploration baselines on the StarCraft Multi-Agent Challenge (SMAC) and SMACv2 environments. Particularly, IIE shows improved performance in the sparse-reward SMAC tasks and produces more effective curricula over the initialized states than other generative methods, such as CVAE-GAN and diffusion models.

Transformer-Based Reinforcement Learning for Multi-Robot Autonomous Exploration

Mapless Collaborative Navigation for a Multi-Robot System Based on the Deep Reinforcement Learning

Efficient Autonomous Exploration of Unknown Environment Using Regions Segmentation and VRP

An Enhanced Hierarchical Planning Framework for Multi-Robot Autonomous Exploration

Asymmetric Information Enhanced Mapping Framework for Multirobot Exploration based on Deep Reinforcement Learning

An improved frontier-based robot exploration strategy combined with deep reinforcement learning

Optimal Exploration Algorithm of Multi-Agent Reinforcement Learning Methods (Student Abstract)

Deep Reinforcement Learning-based Large-scale Robot Exploration

Active Neural Topological Mapping for Multi-Agent Exploration

Leveraging the Efficiency of Multi-Task Robot Manipulation Via Task-Evoked Planner and Reinforcement Learning

A new method of multi-robot cooperative exploration

Control Transformer: Robot Navigation in Unknown Environments through PRM-Guided Return-Conditioned Sequence Modeling

Learning Efficient Multi-Agent Cooperative Visual Exploration

Exploration in Deep Reinforcement Learning: From Single-Agent to Multiagent Domain

DTPPO: Dual-Transformer Encoder-based Proximal Policy Optimization for Multi-UAV Navigation in Unseen Complex Environments

Multi-Robot Environmental Coverage With a Two-Stage Coordination Strategy via Deep Reinforcement Learning

Robot Exploration Based on Small Areas Priority Strategy

Distributed multi-robot potential-field-based exploration with submap-based mapping and noise-augmented strategy

Autonomous and Adaptive Role Selection for Multi-robot Collaborative Area Search Based on Deep Reinforcement Learning

Imagine, Initialize, and Explore: An Effective Exploration Method in Multi-Agent Reinforcement Learning

Topological Map-Based Autonomous Exploration in Large-Scale Scenes for Unmanned Vehicles