Abstract:Sparse reward is one of the most challenging problems in reinforcement learning (RL). Hindsight Experience Replay (HER) attempts to address this issue by converting a failed experience to a successful one by relabelling the goals. In open-ended and changing environments, agents face a wide range of potential tasks that might not come with associated reward functions. Such autonomous learning agents must set their own tasks and build their own curriculum through an intrinsically motivated exploration. Because some tasks might prove easy and some impossible, agents must actively select which task to practice at any given moment, to maximize their overall mastery on the set of learnable tasks. The purpose of this technical report is two-fold. First, it introduces a suite of challenging continuous control tasks (integrated with OpenAI Gym) based on currently existing robotics hardware. The tasks include pushing, sliding and pick & place with a Fetch robotic arm as well as in-hand object manipulation with a Shadow Dexterous Hand. All tasks have sparse binary rewards and follow a Multi-Goal Reinforcement Learning (RL) framework in which an agent is told what to do using an additional input. The second part of the paper presents a set of concrete research ideas for improving RL algorithms, most of which are related to Multi-Goal RL and Hindsight Experience Replay. The Fetch environments are based on the 7-DoF Fetch robotics arm,2 which has a two-fingered parallel gripper. Agents focus on achievable tasks first and focus back on tasks that are being forgotten. Experiments conducted in a new multi-task multi-goal robotic environment show that our algorithm benefits from these two ideas and demonstrate properties of robustness to distracting tasks, forgetting and changes in body properties

Addressing Reward Engineering For Deep Reinforcement Learning On Multi-Stage Task

Achieving Sample-Efficient Learning of Long-Horizon Sparse-Reward Robotic Tasks with Base Controllers

Learning of Long-Horizon Sparse-Reward Robotic Manipulator Tasks With Base Controllers

Data-efficient Deep Reinforcement Learning Method Toward Scaling Continuous Robotic Task with Sparse Rewards.

DrS: Learning Reusable Dense Rewards for Multi-Stage Tasks

Deep Reinforcement Learning for an Anthropomorphic Robotic Arm under Sparse Reward Tasks

Overcoming Exploration in Reinforcement Learning with Demonstrations

Leveraging Demonstrations for Deep Reinforcement Learning on Robotics Problems with Sparse Rewards

Efficient Hindsight Reinforcement Learning Using Demonstrations for Robotic Tasks with Sparse Rewards

Adaptive Reward Design for Reinforcement Learning in Complex Robotic Tasks

Hybrid Reinforcement Learning Based on Human Preference and Advice for Efficient Robot Skill Learning

Tactical Reward Shaping: Bypassing Reinforcement Learning with Strategy-Based Goals

A Study on Dense and Sparse (Visual) Rewards in Robot Policy Learning

Hierarchical Multi-Agent Reinforcement Learning for Cooperative Tasks with Sparse Rewards in Continuous Domain

Task-Oriented Deep Reinforcement Learning for Robotic Skill Acquisition and Control

Intrinsically Motivated Multi-Goal Reinforcement Learning Using Robotics Environment Integrated with OpenAI Gym

Deep Reinforcement Learning with a Stage Incentive Mechanism of Dense Reward for Robotic Trajectory Planning

End-to-End Robotic Reinforcement Learning without Reward Engineering

Revisiting Sparse Rewards for Goal-Reaching Reinforcement Learning

Overcoming Exploration: Deep Reinforcement Learning for Continuous Control in Cluttered Environments from Temporal Logic Specifications

Dense Dynamics-Aware Reward Synthesis: Integrating Prior Experience with Demonstrations