Abstract:Learning from previously collected data via behavioral cloning or offline reinforcement learning (RL) is a powerful recipe for scaling generalist agents by avoiding the need for expensive online learning. Despite strong generalization in some respects, agents are often remarkably brittle to minor visual variations in control-irrelevant factors such as the background or camera viewpoint. In this paper, we present theDeepMind Control Visual Benchmark (DMC-VB), a dataset collected in the DeepMind Control Suite to evaluate the robustness of offline RL agents for solving continuous control tasks from visual input in the presence of visual distractors. In contrast to prior works, our dataset (a) combines locomotion and navigation tasks of varying difficulties, (b) includes static and dynamic visual variations, (c) considers data generated by policies with different skill levels, (d) systematically returns pairs of state and pixel observation, (e) is an order of magnitude larger, and (f) includes tasks with hidden goals. Accompanying our dataset, we propose three benchmarks to evaluate representation learning methods for pretraining, and carry out experiments on several recently proposed methods. First, we find that pretrained representations do not help policy learning on DMC-VB, and we highlight a large representation gap between policies learned on pixel observations and on states. Second, we demonstrate when expert data is limited, policy learning can benefit from representations pretrained on (a) suboptimal data, and (b) tasks with stochastic hidden goals. Our dataset and benchmark code to train and evaluate agents are available at: <a class="link-external link-https" href="https://github.com/google-deepmind/dmc_vision_benchmark" rel="external noopener nofollow">this https URL</a>.

Benchmarking Deep Reinforcement Learning for Continuous Control

RMBench: Benchmarking Deep Reinforcement Learning for Robotic Manipulator Control

Reproducibility of Benchmarked Deep Reinforcement Learning Tasks for Continuous Control

URLB: Unsupervised Reinforcement Learning Benchmark

Deep Model-Based Reinforcement Learning for Predictive Control of Robotic Systems with Dense and Sparse Rewards

Continuous control with deep reinforcement learning

RobocupGym: A challenging continuous control benchmark in Robocup

Using Deep Reinforcement Learning for the Continuous Control of Robotic Arms

DeepMind Control Suite

Benchmarking Safe Exploration in Deep Reinforcement Learning

A survey of benchmarking frameworks for reinforcement learning

Benchmarking Smoothness and Reducing High-Frequency Oscillations in Continuous Control Policies

Multi-task Learning for Continuous Control

Investigating Generalisation in Continuous Deep Reinforcement Learning

safe-control-gym: a Unified Benchmark Suite for Safe Learning-based Control and Reinforcement Learning in Robotics

DMC-VB: A Benchmark for Representation Learning for Control with Visual Distractors

Learnable Behavior Control: Breaking Atari Human World Records via Sample-Efficient Behavior Selection

An Optical Control Environment for Benchmarking Reinforcement Learning Algorithms

Benchmarking Reinforcement Learning Methods for Dexterous Robotic Manipulation with a Three-Fingered Gripper

State of the Art Control of Atari Games Using Shallow Reinforcement Learning

A Survey of Deep Network Solutions for Learning Control in Robotics: From Reinforcement to Imitation