Abstract:In recent years, significant progress has been made in the field of robotic reinforcement learning (RL), enabling methods that handle complex image observations, train in the real world, and incorporate auxiliary data, such as demonstrations and prior experience. However, despite these advances, robotic RL remains hard to use. It is acknowledged among practitioners that the particular implementation details of these algorithms are often just as important (if not more so) for performance as the choice of algorithm. We posit that a significant challenge to widespread adoption of robotic RL, as well as further development of robotic RL methods, is the comparative inaccessibility of such methods. To address this challenge, we developed a carefully implemented library containing a sample efficient off-policy deep RL method, together with methods for computing rewards and resetting the environment, a high-quality controller for a widely-adopted robot, and a number of challenging example tasks. We provide this library as a resource for the community, describe its design choices, and present experimental results. Perhaps surprisingly, we find that our implementation can achieve very efficient learning, acquiring policies for PCB board assembly, cable routing, and object relocation between 25 to 50 minutes of training per policy on average, improving over state-of-the-art results reported for similar tasks in the literature. These policies achieve perfect or near-perfect success rates, extreme robustness even under perturbations, and exhibit emergent recovery and correction behaviors. We hope that these promising results and our high-quality open-source implementation will provide a tool for the robotics community to facilitate further developments in robotic RL. Our code, documentation, and videos can be found at

Model-Based Reinforcement Learning for Robotic Arm Control with Limited Environment Interaction

RL and ANN Based Modular Path Planning Controller for Resource-Constrained Robots in the Indoor Complex Dynamic Environment

Open-Loop Motion Control of a Hydraulic Soft Robotic Arm Using Deep Reinforcement Learning

Learning with Training Wheels: Speeding up Training with a Simple Controller for Deep Reinforcement Learning

Active Exploration in Bayesian Model-based Reinforcement Learning for Robot Manipulation

Learning of Long-Horizon Sparse-Reward Robotic Manipulator Tasks With Base Controllers

Robotic Arm Motion Planning Based on Curriculum Reinforcement Learning

Robotic arm reinforcement learning control method based on autonomous visual perception

Learning Efficient Robot Arm Reaching

Active Exploration Deep Reinforcement Learning for Continuous Action Space with Forward Prediction

Unified Human-Robot-Environment Interaction Control in Contact-Rich Collaborative Manipulation Tasks Via Model-Based Reinforcement Learning

Deep Incremental Model Based Reinforcement Learning: A One-Step Lookback Approach for Continuous Robotics Control

A survey on model-based reinforcement learning

Comparison of Model-Based and Model-Free Reinforcement Learning for Real-World Dexterous Robotic Manipulation Tasks

Robot Navigation with Interaction-based Deep Reinforcement Learning

On-Robot Bayesian Reinforcement Learning for POMDPs

Closed-Loop Long-Horizon Robotic Planning via Equilibrium Sequence Modeling

Active Reinforcement Learning for Robust Building Control

SERL: A Software Suite for Sample-Efficient Robotic Reinforcement Learning

Reinforcement Learning Controllers for Soft Robots using Learned Environments

Adaptive Environment Modeling Based Reinforcement Learning for Collision Avoidance in Complex Scenes