Neural Garment Dynamics via Manifold-Aware Transformers

Peizhuo Li,Tuanfeng Y. Wang,Timur Levent Kesdogan,Duygu Ceylan,Olga Sorkine-Hornung

DOI: https://doi.org/10.1111/cgf.15028

2024-05-13

Abstract:Data driven and learning based solutions for modeling dynamic garments have significantly advanced, especially in the context of digital humans. However, existing approaches often focus on modeling garments with respect to a fixed parametric human body model and are limited to garment geometries that were seen during training. In this work, we take a different approach and model the dynamics of a garment by exploiting its local interactions with the underlying human body. Specifically, as the body moves, we detect local garment-body collisions, which drive the deformation of the garment. At the core of our approach is a mesh-agnostic garment representation and a manifold-aware transformer network design, which together enable our method to generalize to unseen garment and body geometries. We evaluate our approach on a wide variety of garment types and motion sequences and provide competitive qualitative and quantitative results with respect to the state of the art.

Graphics,Computer Vision and Pattern Recognition

What problem does this paper attempt to address?

This paper proposes a new approach to solving the problem of dynamic clothing simulation, especially for clothing, body shapes, and actions that are not present in the training data. Traditional methods usually rely on fixed body models to simulate clothing, limited to the geometric shapes seen in training. In contrast, this work models the dynamics of clothing by utilizing local interactions between clothing and the body. The core is an unstructured representation of clothing and a transformer network with manifold-awareness, enabling the method to generalize to unseen clothing and body geometric shapes. The paper demonstrates qualitative and quantitative results that compete with state-of-the-art methods through evaluations on various clothing types and motion sequences. In summary, the main contributions of this paper are:

1. Introducing a generalizable learning-based approach to predict dynamics of unknown clothing and body types.
2. Introducing a manifold-aware transformer architecture that combines spatial and topological information, robust to changes in underlying grid density such as those caused by cuts.

This method drives clothing deformations by detecting local collisions between clothing and the body during body movements, and encodes dynamic behavior using accumulated features from past frames. The network operates in an autoregressive manner, using past predictions to compute input features for future frames. In this way, the network can predict deformation gradients for each face of the clothing surface and ultimately compute the geometric shape of the next frame of clothing through Poisson solving.