Abstract:Recent communities have seen significant progress in building photo-realistic animatable avatars from sparse multi-view videos. However, current workflows struggle to render realistic garment dynamics for loose-fitting characters as they predominantly rely on naked body models for human modeling while leaving the garment part un-modeled. This is mainly due to that the deformations yielded by loose garments are highly non-rigid, and capturing such deformations often requires dense views as supervision. In this paper, we introduce AniDress, a novel method for generating animatable human avatars in loose clothes using very sparse multi-view videos (4-8 in our setting). To allow the capturing and appearance learning of loose garments in such a situation, we employ a virtual bone-based garment rigging model obtained from physics-based simulation data. Such a model allows us to capture and render complex garment dynamics through a set of low-dimensional bone transformations. Technically, we develop a novel method for estimating temporal coherent garment dynamics from a sparse multi-view video. To build a realistic rendering for unseen garment status using coarse estimations, a pose-driven deformable neural radiance field conditioned on both body and garment motions is introduced, providing explicit control of both parts. At test time, the new garment poses can be captured from unseen situations, derived from a physics-based or neural network-based simulator to drive unseen garment dynamics. To evaluate our approach, we create a multi-view dataset that captures loose-dressed performers with diverse motions. Experiments show that our method is able to render natural garment dynamics that deviate highly from the body and generalize well to both unseen views and poses, surpassing the performance of existing methods. The code and data will be publicly available.

SmPhy: Generating Smooth and Physically Plausible 3D Garment Animations

Motion-Inspired Real-Time Garment Synthesis with Temporal-Consistency

Motion Guided Deep Dynamic 3D Garments

DiffAvatar: Simulation-Ready Garment Optimization with Differentiable Simulation

CaPhy: Capturing Physical Properties for Animatable Human Avatars

Towards Multi-Layered 3D Garments Animation

Automatic temporal clothing deformation prediction: applicable to variable shape and motion

GVPM: Garment Simulation from Video Based on Priori Movements.

Dynamic neural garments

High-Quality Animatable Dynamic Garment Reconstruction from Monocular Videos

GAPS: Geometry-Aware, Physics-Based, Self-Supervised Neural Garment Draping

AniDress: Animatable Loose-Dressed Avatar from Sparse Views Using Garment Rigging Model

GSNet: Generating 3D Garment Animation Via Graph Skinning Network

PERGAMO: Personalized 3D Garments from Monocular Video

Dress Anyone : Automatic Physically-Based Garment Pattern Refitting

Transferring and Fitting Fixed-Sized Garments Onto Bodies of Various Dimensions and Postures

Towards High-Quality 3D Motion Transfer with Realistic Apparel Animation

Dressing Avatars: Deep Photorealistic Appearance for Physically Simulated Clothing

ClotheDreamer: Text-Guided Garment Generation with 3D Gaussians

Garment3DGen: 3D Garment Stylization and Texture Generation

PhysAvatar: Learning the Physics of Dressed 3D Avatars from Visual Observations