Abstract:Recent communities have seen significant progress in building photo-realistic animatable avatars from sparse multi-view videos. However, current workflows struggle to render realistic garment dynamics for loose-fitting characters as they predominantly rely on naked body models for human modeling while leaving the garment part un-modeled. This is mainly due to that the deformations yielded by loose garments are highly non-rigid, and capturing such deformations often requires dense views as supervision. In this paper, we introduce AniDress, a novel method for generating animatable human avatars in loose clothes using very sparse multi-view videos (4-8 in our setting). To allow the capturing and appearance learning of loose garments in such a situation, we employ a virtual bone-based garment rigging model obtained from physics-based simulation data. Such a model allows us to capture and render complex garment dynamics through a set of low-dimensional bone transformations. Technically, we develop a novel method for estimating temporal coherent garment dynamics from a sparse multi-view video. To build a realistic rendering for unseen garment status using coarse estimations, a pose-driven deformable neural radiance field conditioned on both body and garment motions is introduced, providing explicit control of both parts. At test time, the new garment poses can be captured from unseen situations, derived from a physics-based or neural network-based simulator to drive unseen garment dynamics. To evaluate our approach, we create a multi-view dataset that captures loose-dressed performers with diverse motions. Experiments show that our method is able to render natural garment dynamics that deviate highly from the body and generalize well to both unseen views and poses, surpassing the performance of existing methods. The code and data will be publicly available.

GVPM: Garment Simulation from Video Based on Priori Movements.

SmPhy: Generating Smooth and Physically Plausible 3D Garment Animations

Motion Guided Deep Dynamic 3D Garments

GP-VTON: Towards General Purpose Virtual Try-on via Collaborative Local-Flow Global-Parsing Learning

PERGAMO: Personalized 3D Garments from Monocular Video

High-Quality Animatable Dynamic Garment Reconstruction from Monocular Videos

Fashion-VDM: Video Diffusion Model for Virtual Try-On

ViViD: Video Virtual Try-on using Diffusion Models

DiffAvatar: Simulation-Ready Garment Optimization with Differentiable Simulation

Motion-Inspired Real-Time Garment Synthesis with Temporal-Consistency

Generating diverse clothed 3D human animations via a generative model

GSNet: Generating 3D Garment Animation Via Graph Skinning Network

PEMF-VVTO: Point-Enhanced Video Virtual Try-on via Mask-free Paradigm

AniDress: Animatable Loose-Dressed Avatar from Sparse Views Using Garment Rigging Model

Automatic 3D virtual fitting system based on skeleton driving

Gaussian Garments: Reconstructing Simulation-Ready Clothing with Photorealistic Appearance from Multi-View Video

Toward Multi-Category Garments Virtual Try-on Method by Coarse to Fine TPS Deformation

Learning‐Based Animation of Clothing for Virtual Try‐On

Dynamic neural garments

PG-VTON: A Novel Image-Based Virtual Try-On Method Via Progressive Inference Paradigm

Garment Avatars: Realistic Cloth Driving using Pattern Registration