3D Gaussian Model for Animation and Texturing

Xiangzhi Eric Wang,Zackary P. T. Sin

2024-03-01

Abstract:3D Gaussian Splatting has made a marked impact on neural rendering by achieving impressive fidelity and performance. Despite this achievement, however, it is not readily applicable to developing interactive applications. Real-time applications like XR apps and games require functions such as animation, UV-mapping, and model editing simultaneously manipulated through the usage of a 3D model. We propose a modeling that is analogous to typical 3D models, which we call 3D Gaussian Model (3DGM); it provides a manipulatable proxy for novel animation and texture transfer. By binding the 3D Gaussians in texture space and re-projecting them back to world space through implicit shell mapping, we show how our 3D modeling can serve as a valid rendering methodology for interactive applications. It is further noted that recently, 3D mesh reconstruction works have been able to produce high-quality mesh for rendering. Our work, on the other hand, only requires an approximated geometry for rendering an object in high fidelity. Applicationwise, we will show that our proxy-based 3DGM is capable of driving novel animation without animated training data and texture transferring via UV mapping of the 3D Gaussians. We believe the result indicates the potential of our work for enabling interactive applications for 3D Gaussian Splatting.

Graphics

What problem does this paper attempt to address?

### Problems the Paper Attempts to Solve

The paper attempts to address the issue of how to apply 3D Gaussian Splatting (3DGS) to animation and texture transfer problems in interactive applications. Although 3DGS has achieved significant fidelity and performance in neural rendering, it is not directly applicable to the development of interactive applications such as XR applications and games. These applications typically require handling functionalities like animation, UV mapping, and model editing simultaneously.

Specifically, the paper proposes a new method called the 3D Gaussian Model (3DGM), which binds 3D Gaussian points to texture space and reprojects them in world space, providing an operable proxy model for new animation and texture transfer. The main contributions of the paper include:

1. **3D Gaussian Model**: A proxy representation similar to traditional 3D models, suitable for rasterization-based rendering pipelines.
2. **Unidirectional Mapping**: A method for unidirectional mapping from texture space to world space, optimizing 3D Gaussian points without the need to explicitly construct shells.
3. **Training Strategy**: A method to constrain 3D Gaussian points, enabling them to perform animation and texture mapping.

Through these innovations, 3DGM can achieve new animation and texture transfer without the need for animation training data, thereby providing new possibilities for the development of interactive applications.