Abstract:This paper is concerned with learning transferable forward models for push manipulation that can be applying to novel contexts and how to improve the quality of prediction when critical information is available. We propose to learn a parametric internal model for push interactions that, similar for humans, enables a robot to predict the outcome of a physical interaction even in novel contexts. Given a desired push action, humans are capable to identify where to place their finger on a new object so to produce a predictable motion of the object. We achieve the same behaviour by factorising the learning into two parts. First, we learn a set of local contact models to represent the geometrical relations between the robot pusher, the object, and the environment. Then we learn a set of parametric local motion models to predict how these contacts change throughout a push. The set of contact and motion models represent our internal model. By adjusting the shapes of the distributions over the physical parameters, we modify the internal model's response. Uniform distributions yield to coarse estimates when no information is available about the novel context. We call this an unbiased predictor. A more accurate predictor can be learned for a specific environment/object pair (e.g., low friction/high mass), called a biased predictor. The effectiveness of our approach is demonstrated in a simulated environment in which a Pioneer 3-DX robot equipped with a bumper needs to predict a push outcome for an object in a novel context, and we support those results with a proof of concept on a real robot. We train on two objects (a cube and a cylinder) for a total of 24,000 pushes in various conditions, and test on six objects encompassing a variety of shapes, sizes, and physical parameters for a total of 14,400 predicted push outcomes. Our experimental results show that both biased and unbiased predictors can reliably produce predictions in line with the outcomes of a carefully tuned physics simulator.

Object and relation centric representations for push effect prediction

Learning Transferable Push Manipulation Skills in Novel Contexts

UNO Push: Unified Nonprehensile Object Pushing via Non-Parametric Estimation and Model Predictive Control

Learning Goal-Oriented Non-Prehensile Pushing in Cluttered Scenes

Push to know! -- Visuo-Tactile based Active Object Parameter Inference with Dual Differentiable Filtering

Pushing in the Dark: A Reactive Pushing Strategy for Mobile Robots Using Tactile Feedback

Collaborative Planar Pushing of Polytopic Objects with Multiple Robots in Complex Scenes

ReloPush: Multi-object Rearrangement in Confined Spaces with a Nonholonomic Mobile Robot Pusher

Effective and Robust Non-Prehensile Manipulation via Persistent Homology Guided Monte-Carlo Tree Search

Vision Perception-based Adaptive Pushing Assisted Grasping Network for Dense Clutters

Predictive Visuo-Tactile Interactive Perception Framework for Object Properties Inference

Interaction Prediction and Monte-Carlo Tree Search for Robot Manipulation in Clutter

Let's Push Things Forward: A Survey on Robot Pushing

More than a Million Ways to Be Pushed: A High-Fidelity Experimental Dataset of Planar Pushing

Precision-Focused Reinforcement Learning Model for Robotic Object Pushing

Reinforcement Learning With Vision-Proprioception Model for Robot Planar Pushing

Learning Latent Object-Centric Representations for Visual-Based Robot Manipulation

How Physics and Background Attributes Impact Video Transformers in Robotic Manipulation: A Case Study on Planar Pushing

Learning Rearrangement Manipulation Via Scene Prediction in Point Cloud

Pushing Operation of Manipulator Based on Experience Learning: Position Prediction of an Object and Pushing Analysis

Object-Centric Kinodynamic Planning for Nonprehensile Robot Rearrangement Manipulation