Abstract:Text-to-3D generation aims to create 3D assets from text-to-image diffusion models. However, existing methods face an inherent bottleneck in generation quality because the widely-used objectives such as Score Distillation Sampling (SDS) inappropriately omit U-Net jacobians for swift generation, leading to significant bias compared to the "true" gradient obtained by full denoising sampling. This bias brings inconsistent updating direction, resulting in implausible 3D generation e.g., color deviation, Janus problem, and semantically inconsistent details). In this work, we propose Pose-dependent Consistency Distillation Sampling (PCDS), a novel yet efficient objective for diffusion-based 3D generation tasks. Specifically, PCDS builds the pose-dependent consistency function within diffusion trajectories, allowing to approximate true gradients through minimal sampling steps (1-3). Compared to SDS, PCDS can acquire a more accurate updating direction with the same sampling time (1 sampling step), while enabling few-step (2-3) sampling to trade compute for higher generation quality. For efficient generation, we propose a coarse-to-fine optimization strategy, which first utilizes 1-step PCDS to create the basic structure of 3D objects, and then gradually increases PCDS steps to generate fine-grained details. Extensive experiments demonstrate that our approach outperforms the state-of-the-art in generation quality and training efficiency, conspicuously alleviating the implausible 3D generation issues caused by the deviated updating direction. Moreover, it can be simply applied to many 3D generative applications to yield impressive 3D assets, please see our project page: <a class="link-external link-https" href="https://narcissusex.github.io/VividDreamer" rel="external noopener nofollow">this https URL</a>.

Edify 3D: Scalable High-Quality 3D Asset Generation

Paint3D: Paint Anything 3D with Lighting-Less Texture Diffusion Models

Edify Image: High-Quality Image Generation with Pixel Space Laplacian Diffusion Models

Novel 3D-Aware Composition Images Synthesis for Object Display with Diffusion Model.

GraphicsDreamer: Image to 3D Generation with Physical Consistency

High-Fidelity 3D Model Generation with Relightable Appearance from Single Freehand Sketches and Text Guidance

Meta 3D AssetGen: Text-to-Mesh Generation with High-Quality Geometry, Texture, and PBR Materials

BoostDream: Efficient Refining for High-Quality Text-to-3D Generation from Multi-View Diffusion

Magic3D: High-Resolution Text-to-3D Content Creation

One-2-3-45++: Fast Single Image to 3D Objects with Consistent Multi-View Generation and 3D Diffusion

3DTopia-XL: Scaling High-quality 3D Asset Generation via Primitive Diffusion

VividDreamer: Towards High-Fidelity and Efficient Text-to-3D Generation

Flex3D: Feed-Forward 3D Generation With Flexible Reconstruction Model And Input View Curation

Enhanced 3D Generation by 2D Editing

Structured 3D Latents for Scalable and Versatile 3D Generation

HiFi-123: Towards High-fidelity One Image to 3D Content Generation

3D Creation at Your Fingertips: From Text or Image to 3D Assets

En3D: An Enhanced Generative Model for Sculpting 3D Humans from 2D Synthetic Data

GradeADreamer: Enhanced Text-to-3D Generation Using Gaussian Splatting and Multi-View Diffusion

ET3D: Efficient Text-to-3D Generation via Multi-View Distillation

Direct3D: Scalable Image-to-3D Generation via 3D Latent Diffusion Transformer