Abstract:In the rapidly evolving field of reinforcement learning, combination of hierarchical and multiagent learning methods presents unique challenges and opens up new opportunities. This paper discusses a combination of multilevel hierarchical learning with subgoal discovery and multiagent reinforcement learning with hindsight experience replay. Combining these approaches leads to the creation of multiagent subgoal hierarchy algorithm (MASHA) that allows multiple agents to learn efficiently in complex environments, including environments with sparse rewards. We demonstrate the results of the proposed approach in one of these environments inside the StarCraft II strategy game, in addition to making comparisons with other existing approaches. The proposed algorithm is developed in the paradigm of centralized learning with decentralized execution, which makes it possible to achieve a balance between coordination and autonomy of agents.

Hierarchical Method for Cooperative Multiagent Reinforcement Learning in Markov Decision Processes