Abstract:To improve the efficiency of warehousing system and meet huge customer orders, we aim to solve the challenges of dimension disaster and dynamic properties in hyper scale multi-robot task planning (MRTP) for robotic mobile fulfillment system (RMFS). Existing research indicates that hierarchical reinforcement learning (HRL) is an effective method to reduce these challenges. Based on that, we construct an efficient multi-stage HRL-based multi-robot task planner for hyper scale MRTP in RMFS, and the planning process is represented with a special temporal graph topology. To ensure optimality, the planner is designed with a centralized architecture, but it also brings the challenges of scaling up and generalization that require policies to maintain performance for various unlearned scales and maps. To tackle these difficulties, we first construct a hierarchical temporal attention network (HTAN) to ensure basic ability of handling inputs with unfixed lengths, and then design multi-stage curricula for hierarchical policy learning to further improve the scaling up and generalization ability while avoiding catastrophic forgetting. Additionally, we notice that policies with hierarchical structure suffer from unfair credit assignment that is similar to that in multi-agent reinforcement learning, inspired of which, we propose a hierarchical reinforcement learning algorithm with counterfactual rollout baseline to improve learning performance. Experimental results demonstrate that our planner outperform other state-of-the-art methods on various MRTP instances in both simulated and real-world RMFS. Also, our planner can successfully scale up to hyper scale MRTP instances in RMFS with up to 200 robots and 1000 retrieval racks on unlearned maps while keeping superior performance over other methods.

Erlang Planning Network: an Iterative Model-Based Reinforcement Learning with Multi-Perspective

Model-Based Reinforcement Learning with Automated Planning for Network Management

Planning with Exploration: Addressing Dynamics Bottleneck in Model-based Reinforcement Learning

Multiagent Motion Planning Based on Deep Reinforcement Learning in Complex Environments

Network planning with deep reinforcement learning

Multi-robot Social-aware Cooperative Planning in Pedestrian Environments Using Multi-agent Reinforcement Learning

LMRL: a Multi-Agent Reinforcement Learning Model and Algorithm

An Enhanced Hierarchical Planning Framework for Multi-Robot Autonomous Exploration

Model-based Reinforcement Learning for Decentralized Multiagent Rendezvous

Multi-Agent Path Planning Using Deep Reinforcement Learning

Plan to Predict: Learning an Uncertainty-Foreseeing Model for Model-Based Reinforcement Learning.

Neighborhood Curiosity-Based Exploration in Multi-Agent Reinforcement Learning

Graph Neural Networks with Model-based Reinforcement Learning for Multi-agent Systems

Modeling Complex Networks Based on Deep Reinforcement Learning

Improving Planning with Large Language Models: A Modular Agentic Architecture

On the role of planning in model-based deep reinforcement learning

Planning-Augmented Hierarchical Reinforcement Learning

Model-Based Reinforcement Learning Inspired by Augmented PD for Robotic Control

LIRL: Latent Imagination-Based Reinforcement Learning for Efficient Coverage Path Planning

Scalable Hierarchical Reinforcement Learning for Hyper Scale Multi-Robot Task Planning

BiERL: A Meta Evolutionary Reinforcement Learning Framework via Bilevel Optimization