Abstract:Most advances in 3D Generative Adversarial Networks (3D GANs) largely depend on ray casting-based volume rendering, which incurs demanding rendering costs. One promising alternative is rasterization-based 3D Gaussian Splatting (3D-GS), providing a much faster rendering speed and explicit 3D representation. In this paper, we exploit Gaussian as a 3D representation for 3D GANs by leveraging its efficient and explicit characteristics. However, in an adversarial framework, we observe that a naïve generator architecture suffers from training instability and lacks the capability to adjust the scale of Gaussians. This leads to model divergence and visual artifacts due to the absence of proper guidance for initialized positions of Gaussians and densification to manage their scales adaptively. To address these issues, we introduce a generator architecture with a hierarchical multi-scale Gaussian representation that effectively regularizes the position and scale of generated Gaussians. Specifically, we design a hierarchy of Gaussians where finer-level Gaussians are parameterized by their coarser-level counterparts; the position of finer-level Gaussians would be located near their coarser-level counterparts, and the scale would monotonically decrease as the level becomes finer, modeling both coarse and fine details of the 3D scene. Experimental results demonstrate that ours achieves a significantly faster rendering speed (x100) compared to state-of-the-art 3D consistent GANs with comparable 3D generation capability. Project page: <a class="link-external link-https" href="https://hse1032.github.io/gsgan" rel="external noopener nofollow">this https URL</a>.

Learning spectral normalized adversarial systems with stacked structure for high-quality 3D object generation

SDF-based 3D Generative Adversarial Networks for Image and Geometry Generation

3D-Aware Image Synthesis Via Learning Structural and Textural Representations

Learning a Probabilistic Latent Space of Object Shapes via 3D Generative-Adversarial Modeling

SSP-GAN:Spectral Normalization Sphere-Guided 3D Point Cloud Generation

Densely Stacked Generative Adversarial Networks

Stacked Generative Adversarial Networks for Image Generation based on U-Net discriminator

Learning Adversarial 3D Model Generation with 2D Image Enhancer

Two Birds with One Stone: Iteratively Learn Facial Attributes with GANs.

Spectral Image Visualization Using Generative Adversarial Networks

StackGAN++: Realistic Image Synthesis with Stacked Generative Adversarial Networks

MSG-Voxel-GAN: Multi-Scale Gradient Voxel GAN for 3D Object Generation

Distributed Spatio-Temporal Generative Adversarial Networks

GSGAN: Adversarial Learning for Hierarchical Generation of 3D Gaussian Splats

CAD: Photorealistic 3D Generation via Adversarial Distillation

Sparse Representation-Based Face Object Generative Via Deep Adversarial Network

Get3DHuman: Lifting StyleGAN-Human into a 3D Generative Model using Pixel-aligned Reconstruction Priors

Learning Effective NeRFs and SDFs Representations with 3D Generative Adversarial Networks for 3D Object Generation: Technical Report for ICCV 2023 OmniObject3D Challenge

Self-supervised Learning for Enhancing Geometrical Modeling in 3D-Aware Generative Adversarial Network

Spectral Normalization for Generative Adversarial Networks

Constructing an Efficient and Adaptive Learning Model for 3D Object Generation