Abstract:Robotic manipulation of volumetric elastoplastic deformable materials, from foods such as dough to construction materials like clay, is in its infancy, largely due to the difficulty of modelling and perception in a high-dimensional space. Simulating the dynamics of such materials is computationally expensive. It tends to suffer from inaccurately estimated physics parameters of the materials and the environment, impeding high-precision manipulation. Estimating such parameters from raw point clouds captured by optical cameras suffers further from heavy occlusions. To address this challenge, this work introduces a novel Differentiable Physics-based System Identification (DPSI) framework that enables a robot arm to infer the physics parameters of elastoplastic materials and the environment using simple manipulation motions and incomplete 3D point clouds, aligning the simulation with the real world. Extensive experiments show that with only a single real-world interaction, the estimated parameters, Young's modulus, Poisson's ratio, yield stress and friction coefficients, can accurately simulate visually and physically realistic deformation behaviours induced by unseen and long-horizon manipulation motions. Additionally, the DPSI framework inherently provides physically intuitive interpretations for the parameters in contrast to black-box approaches such as deep neural networks.

What problem does this paper attempt to address?

### Problems the paper attempts to solve This paper aims to solve the challenges faced by robots when manipulating volumetric elastoplastic materials such as dough and clay. Specifically, the paper focuses on the following key issues: 1. **Modeling and perception problems**: - When manipulating elastoplastic materials, the deformation dynamics are highly unpredictable due to the materials having an infinite number of degrees of freedom (DoFs). - Conventional robot motion - planning methods usually require explicit physical models, and these models are difficult to apply when dealing with elastoplastic materials in high - dimensional spaces. - Learning methods (such as reinforcement learning) can be used to learn perception and actions, but they are inefficient in practical applications and it is difficult to achieve high - precision manipulation. 2. **Difficulties in physical parameter estimation**: - Simulating the dynamics of such materials is computationally expensive and is easily affected by inaccurate physical parameter estimations. - When estimating physical parameters from the raw point cloud captured by an optical camera, it will be affected by severe occlusion, which further increases the difficulty. 3. **Real - world alignment problems**: - Existing methods are often difficult to align simulation results with the physical behavior in the real world, especially when facing complex and long - term operations. ### Solutions To address the above challenges, the paper proposes a system identification framework based on differentiable physics (DPSI). The main features of this framework are as follows: 1. **Efficient parameter estimation**: - Using simple manipulation actions and incomplete 3D point - cloud data, DPSI can efficiently estimate the physical parameters of the material, such as Young's modulus \( E \), Poisson's ratio \( \nu \), yield stress \( \sigma_y \) and friction coefficient, etc. - Optimize these physical parameters by minimizing the loss function between the simulation results and the real observations. 2. **High - fidelity physical simulation**: - Use the material point method (MPM) for high - fidelity physical simulation, which can efficiently and realistically simulate the deformation behavior of the material. - MPM achieves fast simulation by calculating motion, deformation gradients and frictional contacts on the background Eulerian grid. 3. **Differentiable physics**: - Use the DiffTaiChi programming language to implement differentiable physical simulations, automatically calculate gradients and perform gradient - based optimization. - This allows multiple physical parameters to be directly optimized during the simulation process, thereby improving the alignment accuracy. 4. **Verification in practical applications**: - Through a large number of experimental verifications, DPSI can accurately predict unseen, long - term and complex elastoplastic material manipulation behaviors with only one real - world interaction. ### Summary By proposing the DPSI framework, the paper solves the problems of modeling, parameter estimation and real - world alignment when robots manipulate elastoplastic materials. This framework uses differentiable physical simulation and efficient parameter optimization methods to achieve high - precision material manipulation simulation, providing strong support for practical applications.

Differentiable Physics-based System Identification for Robotic Manipulation of Elastoplastic Materials

Differentiable Cloth Parameter Identification and State Estimation in Manipulation

Parameter Identification of Hydraulic Manipulators Considering Physical Feasibility and Control Stability

Sim2Real^2: Actively Building Explicit Physics Model for Precise Articulated Object Manipulation

Learning physics-informed simulation models for soft robotic manipulation: A case study with dielectric elastomer actuators

Sim2Real<SUP>2</SUP>: Actively Building Explicit Physics Model for Precise Articulated Object Manipulation

Parameter Identification and Motion Control for Articulated Rigid Body Robots Using Differentiable Position-based Dynamics

Sim2Real Neural Controllers for Physics-based Robotic Deployment of Deformable Linear Objects

DIFFTACTILE: A Physics-based Differentiable Tactile Simulator for Contact-rich Robotic Manipulation

Optimal parameter identification for discrete mechanical systems with application to flexible object manipulation

Differentiable Particles for General-Purpose Deformable Object Manipulation

Learning Object Properties Using Robot Proprioception via Differentiable Robot-Object Interaction

Dynamic Parameters Identification for Manipulators Considering Physical Feasibility and Nonlinear Friction.

RoboPack: Learning Tactile-Informed Dynamics Models for Dense Packing

Contact Points Discovery for Soft-Body Manipulations with Differentiable Physics

Robust Dexterous Manipulation Under Object Dynamics Uncertainties

Achieving Autonomous Cloth Manipulation with Optimal Control via Differentiable Physics-Aware Regularization and Safety Constraints

An Approach for Identifying Dynamic Parameters in Robotic Systems With Inconsistent Joint Measurements

Soft Contact Simulation and Manipulation Learning of Deformable Objects with Vision-based Tactile Sensor

LLM for Differentiable Soft Robotic Vision Objects Manipulation

Thin-Shell Object Manipulations With Differentiable Physics Simulations