Hyper-3DG: Text-to-3D Gaussian Generation via Hypergraph

Donglin Di,Jiahui Yang,Chaofan Luo,Zhou Xue,Wei Chen,Xun Yang,Yue Gao

2024-03-14

Abstract:Text-to-3D generation represents an exciting field that has seen rapid advancements, facilitating the transformation of textual descriptions into detailed 3D models. However, current progress often neglects the intricate high-order correlation of geometry and texture within 3D objects, leading to challenges such as over-smoothness, over-saturation and the Janus problem. In this work, we propose a method named ``3D Gaussian Generation via Hypergraph (Hyper-3DG)'', designed to capture the sophisticated high-order correlations present within 3D objects. Our framework is anchored by a well-established mainflow and an essential module, named ``Geometry and Texture Hypergraph Refiner (HGRefiner)''. This module not only refines the representation of 3D Gaussians but also accelerates the update process of these 3D Gaussians by conducting the Patch-3DGS Hypergraph Learning on both explicit attributes and latent visual features. Our framework allows for the production of finely generated 3D objects within a cohesive optimization, effectively circumventing degradation. Extensive experimentation has shown that our proposed method significantly enhances the quality of 3D generation while incurring no additional computational overhead for the underlying framework. (Project code:

Computer Vision and Pattern Recognition

What problem does this paper attempt to address?

### The Problem the Paper Attempts to Solve

This paper aims to address the issue of high-order correlations in the text-to-3D generation process, particularly the complex relationships between geometry and texture in 3D objects. Existing methods often overlook these high-order correlations, resulting in generated 3D models that suffer from over-smoothing, over-saturation, inconsistency, and Janus problems.

To overcome these issues, the research proposes a method called "Hyper-3DG: 3D Gaussian Generation via Hypergraph." This framework includes the following key points:

1. **Mainflow**: Utilizes a pre-trained 3D generator and a 2D diffusion model to initialize the generation of rough 3D objects.
2. **HGRefiner**: Refines the geometric and textural details of the generated 3D objects through patch-level 3D Gaussian hypergraph learning, thereby improving generation quality while maintaining computational efficiency.

This method not only significantly enhances the quality of 3D generation but also does not add extra computational overhead to the basic framework. Experimental results show that compared to existing methods, Hyper-3DG excels in cross-view consistency, color and texture realism, and structural integrity.