Abstract:Offline reinforcement learning algorithms hold the promise of enabling data-driven RL methods that do not require costly or dangerous real-world exploration and benefit from large pre-collected datasets. This in turn can facilitate real-world applications, as well as a more standardized approach to RL research. Furthermore, offline RL methods can provide effective initializations for online finetuning to overcome challenges with exploration. However, evaluating progress on offline RL algorithms requires effective and challenging benchmarks that capture properties of real-world tasks, provide a range of task difficulties, and cover a range of challenges both in terms of the parameters of the domain (e.g., length of the horizon, sparsity of rewards) and the parameters of the data (e.g., narrow demonstration data or broad exploratory data). While considerable progress in offline RL in recent years has been enabled by simpler benchmark tasks, the most widely used datasets are increasingly saturating in performance and may fail to reflect properties of realistic tasks. We propose a new benchmark for offline RL that focuses on realistic simulations of robotic manipulation and locomotion environments, based on models of real-world robotic systems, and comprising a variety of data sources, including scripted data, play-style data collected by human teleoperators, and other data sources. Our proposed benchmark covers state-based and image-based domains, and supports both offline RL and online fine-tuning evaluation, with some of the tasks specifically designed to require both pre-training and fine-tuning. We hope that our proposed benchmark will facilitate further progress on both offline RL and fine-tuning algorithms. Website with code, examples, tasks, and data is available at \url{<a class="link-external link-https" href="https://sites.google.com/view/d5rl/" rel="external noopener nofollow">this https URL</a>}

URLB: Unsupervised Reinforcement Learning Benchmark

Benchmarking Deep Reinforcement Learning for Continuous Control

Benchmarking Model-Based Reinforcement Learning

Benchmarking Reinforcement Learning Techniques for Autonomous Navigation

D5RL: Diverse Datasets for Data-Driven Deep Reinforcement Learning

A survey of benchmarking frameworks for reinforcement learning

Open RL Benchmark: Comprehensive Tracked Experiments for Reinforcement Learning

Benchmarking Reinforcement Learning Algorithms on Real-World Robots

RobocupGym: A challenging continuous control benchmark in Robocup

Challenges of real-world reinforcement learning: definitions, benchmarks and analysis

D4RL: Datasets for Deep Data-Driven Reinforcement Learning

RMBench: Benchmarking Deep Reinforcement Learning for Robotic Manipulator Control

Reinforcement Learning with Unsupervised Auxiliary Tasks

RRLS : Robust Reinforcement Learning Suite

Uni-RLHF: Universal Platform and Benchmark Suite for Reinforcement Learning with Diverse Human Feedback

Benchmarking Safe Exploration in Deep Reinforcement Learning

Datasets and Benchmarks for Offline Safe Reinforcement Learning

Automatic Curriculum Learning For Deep RL: A Short Survey

FightLadder: A Benchmark for Competitive Multi-Agent Reinforcement Learning

Challenges and Opportunities in Offline Reinforcement Learning from Visual Observations

DeepMind Control Suite