Abstract:Learning goal conditioned control in the real world is a challenging open problem in robotics. Reinforcement learning systems have the potential to learn autonomously via trial-and-error, but in practice the costs of manual reward design, ensuring safe exploration, and hyperparameter tuning are often enough to preclude real world deployment. Imitation learning approaches, on the other hand, offer a simple way to learn control in the real world, but typically require costly curated demonstration data and lack a mechanism for continuous improvement. Recently, iterative imitation techniques have been shown to learn goal directed control from undirected demonstration data, and improve continuously via self-supervised goal reaching, but results thus far have been limited to simulated environments. In this work, we present evidence that iterative imitation learning can scale to goal-directed behavior on a real robot in a dynamic setting: high speed, precision table tennis (e.g. "land the ball on this particular target"). We find that this approach offers a straightforward way to do continuous on-robot learning, without complexities such as reward design or sim-to-real transfer. It is also scalable -- sample efficient enough to train on a physical robot in just a few hours. In real world evaluations, we find that the resulting policy can perform on par or better than amateur humans (with players sampled randomly from a robotics lab) at the task of returning the ball to specific targets on the table. Finally, we analyze the effect of an initial undirected bootstrap dataset size on performance, finding that a modest amount of unstructured demonstration data provided up-front drastically speeds up the convergence of a general purpose goal-reaching policy. See <a class="link-external link-https" href="https://sites.google.com/view/goals-eye" rel="external noopener nofollow">this https URL</a> for videos.

Learning Efficient Representations for Goal-conditioned Reinforcement Learning Via Tabu Search

Learning Robot Manipulation Skills from Human Demonstration Videos Using Two-Stream 2-D/3-D Residual Networks with Self-Attention

Learning Hierarchical Graph-Based Policy for Goal-Reaching in Unknown Environments

Weakly Supervised Disentangled Representation for Goal-conditioned Reinforcement Learning

GoalsEye: Learning High Speed Precision Table Tennis on a Physical Robot

Adaptive goal selection for agents in dynamic environments

Goal exploration augmentation via pre-trained skills for sparse-reward long-horizon goal-conditioned reinforcement learning

Learning Object Relation Graph and Goal-Aware Policy for Visual Room Rearrangement

Learn Goal-Conditioned Policy with Intrinsic Motivation for Deep Reinforcement Learning

Incorporating Task Progress Knowledge for Subgoal Generation in Robotic Manipulation through Image Edits

Learning Manipulation Skills through Robot Chain-of-Thought with Sparse Failure Guidance

A data-efficient goal-directed deep reinforcement learning method for robot visuomotor skill

Robot in a China Shop: Using Reinforcement Learning for Location-Specific Navigation Behaviour

A Fully Controllable Agent in the Path Planning using Goal-Conditioned Reinforcement Learning

HOGN-TVGN: Human-inspired Embodied Object Goal Navigation Based on Time-varying Knowledge Graph Inference Networks for Robots

Robot Subgoal-guided Navigation in Dynamic Crowded Environments with Hierarchical Deep Reinforcement Learning

SGL: Symbolic Goal Learning in a Hybrid, Modular Framework for Human Instruction Following

Goal-Conditioned Reinforcement Learning with Disentanglement-based Reachability Planning

State-Dependent Maximum Entropy Reinforcement Learning for Robot Long-Horizon Task Learning

Unsupervised Visuomotor Control through Distributional Planning Networks

Generating Attentive Goals for Prioritized Hindsight Reinforcement Learning