Abstract:Recent advancements in reinforcement learning have made significant impacts across various domains, yet they often struggle in complex multi-agent environments due to issues like algorithm instability, low sampling efficiency, and the challenges of exploration and dimensionality explosion. Hierarchical reinforcement learning (HRL) offers a structured approach to decompose complex tasks into simpler sub-tasks, which is promising for multi-agent settings. This paper advances the field by introducing a hierarchical architecture that autonomously generates effective subgoals without explicit constraints, enhancing both flexibility and stability in training. We propose a dynamic goal generation strategy that adapts based on environmental changes. This method significantly improves the adaptability and sample efficiency of the learning process. Furthermore, we address the critical issue of credit assignment in multi-agent systems by synergizing our hierarchical architecture with a modified QMIX network, thus improving overall strategy coordination and efficiency. Comparative experiments with mainstream reinforcement learning algorithms demonstrate the superior convergence speed and performance of our approach in both single-agent and multi-agent environments, confirming its effectiveness and flexibility in complex scenarios. Our code is open-sourced at: \url{<a class="link-external link-https" href="https://github.com/SICC-Group/GMAH" rel="external noopener nofollow">this https URL</a>}.

Past Data-Driven Adaptation in Hierarchical Reinforcement Learning

Advances in Hierarchical Reinforcement Learning

Hierarchical Reinforcement Learning with Advantage-Based Auxiliary Rewards

Causality-driven Hierarchical Structure Discovery for Reinforcement Learning

Temporal-adaptive Hierarchical Reinforcement Learning

Deep Reinforcement Learning with Hierarchical Structures.

HCS-R-HER: Hierarchical Reinforcement Learning Based on Cross Subtasks Rainbow Hindsight Experience Replay

Data-Efficient Hierarchical Reinforcement Learning for Robotic Assembly Control Applications

Hierarchical Reinforcement Learning with Attention Reward

Sub-policy Adaptation for Hierarchical Reinforcement Learning

Why Does Hierarchy (Sometimes) Work So Well in Reinforcement Learning?

On Credit Assignment in Hierarchical Reinforcement Learning

Subgoal-based Hierarchical Reinforcement Learning for Multi-Agent Collaboration

Hierarchical Reinforcement Learning for Temporal Pattern Prediction

Hierarchical Reinforcement Learning in Complex 3D Environments

Abstract Value Iteration for Hierarchical Reinforcement Learning

Towards Efficient Long-Horizon Decision-Making Using Automated Structure Search Method of Hierarchical Reinforcement Learning for Edge Artificial Intelligence

Deep Reinforcement Learning from Hierarchical Preference Design

Hierarchical reinforcement learning with natural language subgoals

Dynamic Hierarchies in Hierarchical Reinforcement Learning