Abstract:Meta-reinforcement learning algorithms can enable robots to acquire new skills much more quickly, by leveraging prior experience to learn how to learn. However, much of the current research on meta-reinforcement learning focuses on task distributions that are very narrow. For example, a commonly used meta-reinforcement learning benchmark uses different running velocities for a simulated robot as different tasks. When policies are meta-trained on such narrow task distributions, they cannot possibly generalize to more quickly acquire entirely new tasks. Therefore, if the aim of these methods is to enable faster acquisition of entirely new behaviors, we must evaluate them on task distributions that are sufficiently broad to enable generalization to new behaviors. In this paper, we propose an open-source simulated benchmark for meta-reinforcement learning and multi-task learning consisting of 50 distinct robotic manipulation tasks. Our aim is to make it possible to develop algorithms that generalize to accelerate the acquisition of entirely new, held-out tasks. We evaluate 7 state-of-the-art meta-reinforcement learning and multi-task learning algorithms on these tasks. Surprisingly, while each task and its variations (e.g., with different object positions) can be learned with reasonable success, these algorithms struggle to learn with multiple tasks at the same time, even with as few as ten distinct training tasks. Our analysis and open-source environments pave the way for future research in multi-task learning and meta-learning that can enable meaningful generalization, thereby unlocking the full potential of these methods.

MGHRL: Meta Goal-generation for Hierarchical Reinforcement Learning

Hierarchical Meta-Reinforcement Learning via Automated Macro-Action Discovery

Meta-Learning Integration in Hierarchical Reinforcement Learning for Advanced Task Complexity

Efficient Hierarchical Exploration with an Active Subgoal Generation Strategy.

Subgoal-based Hierarchical Reinforcement Learning for Multi-Agent Collaboration

HMRL: Hyper-Meta Learning for Sparse Reward Reinforcement Learning Problem

Guided Meta-Policy Search

MetaCURE: Meta Reinforcement Learning with Empowerment-Driven Exploration

Guided Goal Generation for Hindsight Multi-Goal Reinforcement Learning

Prediction Guided Meta-Learning for Multi-Objective Reinforcement Learning

Active Hierarchical Exploration with Stable Subgoal Representation Learning

Hierarchical reinforcement learning for handling sparse rewards in multi-goal navigation

Generating Adjacency-Constrained Subgoals in Hierarchical Reinforcement Learning

Probabilistic Subgoal Representations for Hierarchical Reinforcement learning

Meta-Reinforcement Learning with Universal Policy Adaptation: Provable Near-Optimality under All-task Optimum Comparator

Screening Goals and Selecting Policies in Hierarchical Reinforcement Learning.

Exploration With Task Information for Meta Reinforcement Learning

Model-based Adversarial Meta-Reinforcement Learning

Learning Meta Representations for Agents in Multi-Agent Reinforcement Learning

A Novel Generalized Meta Hierarchical Reinforcement Learning Method for Autonomous Vehicles

Meta-World: A Benchmark and Evaluation for Multi-Task and Meta Reinforcement Learning