Abstract:In this paper, we develop a new method, termed SDF-3DGAN, for 3D object generation and 3D-Aware image synthesis tasks, which introduce implicit Signed Distance Function (SDF) as the 3D object representation method in the generative field. We apply SDF for higher quality representation of 3D object in space and design a new SDF neural renderer, which has higher efficiency and higher accuracy. To train only on 2D images, we first generate the objects, which are represented by SDF, from Gaussian distribution. Then we render them to 2D images and use them to apply GAN training method together with 2D images in the dataset. In the new rendering method, we relieve all the potential of SDF mathematical property to alleviate computation pressure in the previous SDF neural renderer. In specific, our new SDF neural renderer can solve the problem of sampling ambiguity when the number of sampling point is not enough, \ie use the less points to finish higher quality sampling task in the rendering pipeline. And in this rendering pipeline, we can locate the surface easily. Therefore, we apply normal loss on it to control the smoothness of generated object surface, which can make our method enjoy the much higher generation quality. Quantitative and qualitative experiments conducted on public benchmarks demonstrate favorable performance against the state-of-the-art methods in 3D object generation task and 3D-Aware image synthesis task. Our codes will be released at <a class="link-external link-https" href="https://github.com/lutao2021/SDF-3DGAN" rel="external noopener nofollow">this https URL</a>.

SDF-based 3D Generative Adversarial Networks for Image and Geometry Generation

3D-Aware Image Synthesis Via Learning Structural and Textural Representations

SpatialGAN: Progressive Image Generation Based on Spatial Recursive Adversarial Expansion

GeoGen: Geometry-Aware Generative Modeling via Signed Distance Functions

StyleSDF: High-Resolution 3D-Consistent Image and Geometry Generation

SDF‐StyleGAN: Implicit SDF‐Based StyleGAN for 3D Shape Generation

SDF-3DGAN: A 3D Object Generative Method Based on Implicit Signed Distance Function

Latent-Space Laplacian Pyramids for Adversarial Representation Learning with 3D Point Clouds

XDGAN: Multi-Modal 3D Shape Generation in 2D Space

GET3D: A Generative Model of High Quality 3D Textured Shapes Learned from Images

Geometry aware 3D generation from in-the-wild images in ImageNet

What You See is What You GAN: Rendering Every Pixel for High-Fidelity Geometry in 3D GANs

Multi-View Consistent Generative Adversarial Networks for 3D-Aware Image Synthesis

Learning Adversarial 3D Model Generation with 2D Image Enhancer

3D Design Using Generative Adversarial Networks and Physics-Based Validation

GSDF: 3DGS Meets SDF for Improved Rendering and Reconstruction

CAD: Photorealistic 3D Generation via Adversarial Distillation

Learning Effective NeRFs and SDFs Representations with 3D Generative Adversarial Networks for 3D Object Generation: Technical Report for ICCV 2023 OmniObject3D Challenge

Graphic Design of 3D Animation Scenes Based on Deep Learning and Information Security Technology

Locally Attentional SDF Diffusion for Controllable 3D Shape Generation

A Variational Autoencoder Cascade Generative Adversarial Network for Scalable 3D Object Generation and Reconstruction