LAGA: Layered 3D Avatar Generation and Customization via Gaussian Splatting

Jia Gong,Shenyu Ji,Lin Geng Foo,Kang Chen,Hossein Rahmani,Jun Liu

2024-05-21

Abstract:Creating and customizing a 3D clothed avatar from textual descriptions is a critical and challenging task. Traditional methods often treat the human body and clothing as inseparable, limiting users' ability to freely mix and match garments. In response to this limitation, we present LAyered Gaussian Avatar (LAGA), a carefully designed framework enabling the creation of high-fidelity decomposable avatars with diverse garments. By decoupling garments from avatar, our framework empowers users to conviniently edit avatars at the garment level. Our approach begins by modeling the avatar using a set of Gaussian points organized in a layered structure, where each layer corresponds to a specific garment or the human body itself. To generate high-quality garments for each layer, we introduce a coarse-to-fine strategy for diverse garment generation and a novel dual-SDS loss function to maintain coherence between the generated garments and avatar components, including the human body and other garments. Moreover, we introduce three regularization losses to guide the movement of Gaussians for garment transfer, allowing garments to be freely transferred to various avatars. Extensive experimentation demonstrates that our approach surpasses existing methods in the generation of 3D clothed humans.

Graphics,Computer Vision and Pattern Recognition

What problem does this paper attempt to address?

The problem that this paper attempts to solve is how to achieve independent generation and editing of character clothing when generating 3D virtual avatars, thereby allowing users to freely mix and match different clothes to create new variants. Traditional 3D virtual character generation methods usually regard the human body and clothing as an indivisible whole, which limits the user's ability to customize, especially when it is necessary to decorate a specific character with diverse clothes and accessories, such as applications in games or virtual reality environments.

To overcome this challenge, the paper proposes the LAyered Gaussian Avatar (LAGA) framework, which achieves high - fidelity layered virtual character generation through Gaussian Splatting (GS) technology. The main features of the LAGA framework are as follows:

1. **Layered structure**: LAGA decomposes a virtual character wearing clothes into multiple layers, each layer corresponding to a specific piece of clothing or the human body itself, so that users can conveniently edit virtual characters at the clothing level and create new variations.

2. **High - quality clothing generation**: In order to generate high - quality clothing, LAGA introduces a coarse - to - fine strategy to generate diverse clothes and proposes a new dual - SDS loss function to maintain the consistency of the generated clothing with other parts of the virtual character, including the human body and other clothes.

3. **Flexibility of clothing transfer**: LAGA also introduces three regularization losses to guide the movement of Gaussian points, so as to freely transfer clothes between different human body shapes, solving the problem that it is difficult to adapt to different body types in traditional methods due to the close coupling between clothes and the human body.

Through these designs, LAGA can not only generate high - quality 3D virtual characters, but also support easy adaptation and transfer of clothes between different human body shapes, greatly improving the customizability and flexibility of virtual characters.