Abstract:We present a data-driven method for deformation capture and modeling of general soft objects. We adopt an iterative framework that consists of one component for physics-based deformation tracking and another for spacetime optimization of deformation parameters. Low cost depth sensors are used for the deformation capture, and we do not require any force-displacement measurements, thus making the data capture a cheap and convenient process. We augment a state-of-the-art probabilistic tracking method to robustly handle noise, occlusions, fast movements and large deformations. The spacetime optimization aims to match the simulated trajectories with the tracked ones. The optimized deformation model is then used to boost the accuracy of the tracking results, which can in turn improve the deformation parameter estimation itself in later iterations. Numerical experiments demonstrate that the tracking and parameter optimization components complement each other nicely. Our spacetime optimization of the deformation model includes not only the material elasticity parameters and dynamic damping coefficients, but also the reference shape which can differ significantly from the static shape for soft objects. The resulting optimization problem is highly nonlinear in high dimensions, and challenging to solve with previous methods. We propose a novel splitting algorithm that alternates between reference shape optimization and deformation parameter estimation, and thus enables tailoring the optimization of each subproblem more efficiently and robustly. Our system enables realistic motion reconstruction as well as synthesis of virtual soft objects in response to user stimulation. Validation experiments show that our method not only is accurate, but also compares favorably to existing techniques. We also showcase the ability of our system with high quality animations generated from optimized deformation parameters for a variety of soft objects, such as live plants and fabricated models.

Realtime Simulation of Thin-Shell Deformable Materials using CNN-Based Mesh Embedding

D-Cloth: Skinning-based Cloth Dynamic Prediction with a Three-stage Network

NeuralClothSim: Neural Deformation Fields Meet the Thin Shell Theory

Skinning a Parameterization of Three-Dimensional Space for Neural Network Cloth

Deep Deformation Detail Synthesis for Thin Shell Models

Accelerate Neural Subspace-Based Reduced-Order Solver of Deformable Simulation by Lipschitz Optimization

A Physics-embedded Deep Learning Framework for Cloth Simulation

Exploring the potential of transfer learning for metamodels of heterogeneous material deformation

N-Cloth: Predicting 3D Cloth Deformation with Mesh-Based Networks

Convolution, aggregation and attention based deep neural networks for accelerating simulations in mechanics

Neural Material: Learning Elastic Constitutive Material and Damping Models from Sparse Data

Residual MeshNet: Learning to Deform Meshes for Single-View 3D Reconstruction

Diffeomorphic Mesh Deformation via Efficient Optimal Transport for Cortical Surface Reconstruction

Deep Physics-aware Inference of Cloth Deformation for Monocular Human Performance Capture

Image-based deep learning of finite element simulations for fast surrogate biomechanical organ deformations

Deformation capture and modeling of soft objects

Efficient mapping between void shapes and stress fields using Deep Convolutional Neural Networks with Sparse Data

DeepPhysics: a physics aware deep learning framework for real-time simulation

CTSN: Predicting Cloth Deformation for Skeleton-based Characters with a Two-stream Skinning Network

Efficient and Flexible Deformation Representation for Data-Driven Surface Modeling

Multi-Feature Super-Resolution Network for Cloth Wrinkle Synthesis