Abstract:To improve the efficiency of warehousing system and meet huge customer orders, we aim to solve the challenges of dimension disaster and dynamic properties in hyper scale multi-robot task planning (MRTP) for robotic mobile fulfillment system (RMFS). Existing research indicates that hierarchical reinforcement learning (HRL) is an effective method to reduce these challenges. Based on that, we construct an efficient multi-stage HRL-based multi-robot task planner for hyper scale MRTP in RMFS, and the planning process is represented with a special temporal graph topology. To ensure optimality, the planner is designed with a centralized architecture, but it also brings the challenges of scaling up and generalization that require policies to maintain performance for various unlearned scales and maps. To tackle these difficulties, we first construct a hierarchical temporal attention network (HTAN) to ensure basic ability of handling inputs with unfixed lengths, and then design multi-stage curricula for hierarchical policy learning to further improve the scaling up and generalization ability while avoiding catastrophic forgetting. Additionally, we notice that policies with hierarchical structure suffer from unfair credit assignment that is similar to that in multi-agent reinforcement learning, inspired of which, we propose a hierarchical reinforcement learning algorithm with counterfactual rollout baseline to improve learning performance. Experimental results demonstrate that our planner outperform other state-of-the-art methods on various MRTP instances in both simulated and real-world RMFS. Also, our planner can successfully scale up to hyper scale MRTP instances in RMFS with up to 200 robots and 1000 retrieval racks on unlearned maps while keeping superior performance over other methods.

Scaling intelligent agent combat behaviors through hierarchical reinforcement learning

Scaling Intelligent Agents in Combat Simulations for Wargaming

Mastering the Digital Art of War: Developing Intelligent Combat Simulation Agents for Wargaming Using Hierarchical Reinforcement Learning

Scaling Artificial Intelligence for Digital Wargaming in Support of Decision-Making

Hierarchical Multi-Agent Reinforcement Learning for Air Combat Maneuvering

Curriculum-RL Based Air Combat Decision-Making

Emergent Solutions to High-Dimensional Multitask Reinforcement Learning

Large Scale Deep Reinforcement Learning in War-games.

Hierarchical Reinforcement Learning for Air-to-Air Combat

Advances in hierarchical reinforcement learning

Room Clearance with Feudal Hierarchical Reinforcement Learning

Hierarchical Reinforcement Learning from Competitive Self-play for Dual-aircraft formation air combat

Multi-agent Cooperative Combat Simulation in Naval Battlefield with Reinforcement Learning

Generating intelligent agent behaviors in multi-agent game AI using deep reinforcement learning algorithm

Hierarchical Reinforcement Learning in Complex 3D Environments

Deep Reinforcement-Learning-Based Air-Combat-Maneuver Generation Framework

Human-AI Collaboration in Real-World Complex Environment with Reinforcement Learning

Multi-Agent Air Combat Decision-making Based on Battlefield Attention Information

Intelligent air defense task assignment based on hierarchical reinforcement learning

Scalable Hierarchical Reinforcement Learning for Hyper Scale Multi-Robot Task Planning