Abstract:Learning goal conditioned control in the real world is a challenging open problem in robotics. Reinforcement learning systems have the potential to learn autonomously via trial-and-error, but in practice the costs of manual reward design, ensuring safe exploration, and hyperparameter tuning are often enough to preclude real world deployment. Imitation learning approaches, on the other hand, offer a simple way to learn control in the real world, but typically require costly curated demonstration data and lack a mechanism for continuous improvement. Recently, iterative imitation techniques have been shown to learn goal directed control from undirected demonstration data, and improve continuously via self-supervised goal reaching, but results thus far have been limited to simulated environments. In this work, we present evidence that iterative imitation learning can scale to goal-directed behavior on a real robot in a dynamic setting: high speed, precision table tennis (e.g. "land the ball on this particular target"). We find that this approach offers a straightforward way to do continuous on-robot learning, without complexities such as reward design or sim-to-real transfer. It is also scalable -- sample efficient enough to train on a physical robot in just a few hours. In real world evaluations, we find that the resulting policy can perform on par or better than amateur humans (with players sampled randomly from a robotics lab) at the task of returning the ball to specific targets on the table. Finally, we analyze the effect of an initial undirected bootstrap dataset size on performance, finding that a modest amount of unstructured demonstration data provided up-front drastically speeds up the convergence of a general purpose goal-reaching policy. See <a class="link-external link-https" href="https://sites.google.com/view/goals-eye" rel="external noopener nofollow">this https URL</a> for videos.

Learning to Play Table Tennis From Scratch Using Muscular Robots

Towards High Level Skill Learning: Learn to Return Table Tennis Ball Using Monte-Carlo Based Policy Gradient Method.

Optimal stroke learning with policy gradient approach for robotic table tennis

GoalsEye: Learning High Speed Precision Table Tennis on a Physical Robot

Learning to Play Precision Ball Sports from scratch: a Deep Reinforcement Learning Approach

Robotic Table Tennis: A Case Study into a High Speed Learning System

Sample-efficient Reinforcement Learning in Robotic Table Tennis

Ball Motion Control in the Table Tennis Robot System Using Time-Series Deep Reinforcement Learning

Stylized Table Tennis Robots Skill Learning with Incomplete Human Demonstrations

Achieving Human Level Competitive Robot Table Tennis

Robot Air Hockey: A Manipulation Testbed for Robot Learning with Reinforcement Learning

Learning Agile Soccer Skills for a Bipedal Robot with Deep Reinforcement Learning

A Model-free Approach to Stroke Learning for Robotic Table Tennis

Learning to Control Highly Accelerated Ballistic Movements on Muscular Robots

Training Musculoskeletal Arm Play Taichi with Deep Reinforcement Learning

Learning to Play Air Hockey with Model-Based Deep Reinforcement Learning

Learning Diverse Robot Striking Motions with Diffusion Models and Kinematically Constrained Gradient Guidance

A Retrospective on the Robot Air Hockey Challenge: Benchmarking Robust, Reliable, and Safe Learning Techniques for Real-world Robotics

Reinforcement Learning Enabled Automatic Impedance Control of a Robotic Knee Prosthesis to Mimic the Intact Knee Motion in a Co-Adapting Environment

Safe Table Tennis Swing Stroke with Low-Cost Hardware

i-Sim2Real: Reinforcement Learning of Robotic Policies in Tight Human-Robot Interaction Loops