Abstract:Text-to-3D generation aims to create 3D assets from text-to-image diffusion models. However, existing methods face an inherent bottleneck in generation quality because the widely-used objectives such as Score Distillation Sampling (SDS) inappropriately omit U-Net jacobians for swift generation, leading to significant bias compared to the "true" gradient obtained by full denoising sampling. This bias brings inconsistent updating direction, resulting in implausible 3D generation e.g., color deviation, Janus problem, and semantically inconsistent details). In this work, we propose Pose-dependent Consistency Distillation Sampling (PCDS), a novel yet efficient objective for diffusion-based 3D generation tasks. Specifically, PCDS builds the pose-dependent consistency function within diffusion trajectories, allowing to approximate true gradients through minimal sampling steps (1-3). Compared to SDS, PCDS can acquire a more accurate updating direction with the same sampling time (1 sampling step), while enabling few-step (2-3) sampling to trade compute for higher generation quality. For efficient generation, we propose a coarse-to-fine optimization strategy, which first utilizes 1-step PCDS to create the basic structure of 3D objects, and then gradually increases PCDS steps to generate fine-grained details. Extensive experiments demonstrate that our approach outperforms the state-of-the-art in generation quality and training efficiency, conspicuously alleviating the implausible 3D generation issues caused by the deviated updating direction. Moreover, it can be simply applied to many 3D generative applications to yield impressive 3D assets, please see our project page: <a class="link-external link-https" href="https://narcissusex.github.io/VividDreamer" rel="external noopener nofollow">this https URL</a>.

DreamBeast: Distilling 3D Fantastical Animals with Part-Aware Knowledge Transfer

BoostDream: Efficient Refining for High-Quality Text-to-3D Generation from Multi-View Diffusion

StableDreamer: Taming Noisy Score Distillation Sampling for Text-to-3D

YouDream: Generating Anatomically Controllable Consistent Text-to-3D Animals

DreamCreature: Crafting Photorealistic Virtual Creatures from Imagination

Grounded Compositional and Diverse Text-to-3D with Pretrained Multi-View Diffusion Model

VividDreamer: Towards High-Fidelity and Efficient Text-to-3D Generation

ProlificDreamer: High-Fidelity and Diverse Text-to-3D Generation with Variational Score Distillation

Farm3D: Learning Articulated 3D Animals by Distilling 2D Diffusion

DreamCatalyst: Fast and High-Quality 3D Editing via Controlling Editability and Identity Preservation

AnimatableDreamer: Text-Guided Non-rigid 3D Model Generation and Reconstruction with Canonical Score Distillation

Retrieval-Augmented Score Distillation for Text-to-3D Generation

ScaleDreamer: Scalable Text-to-3D Synthesis with Asynchronous Score Distillation

DreamWaltz: Make a Scene with Complex 3D Animatable Avatars

DreamWaltz-G: Expressive 3D Gaussian Avatars from Skeleton-Guided 2D Diffusion

VividDreamer: Invariant Score Distillation For Hyper-Realistic Text-to-3D Generation

DreamPolish: Domain Score Distillation with Progressive Geometry Generation

EfficientDreamer: High-Fidelity and Robust 3D Creation via Orthogonal-view Diffusion Prior

Semantic Score Distillation Sampling for Compositional Text-to-3D Generation

PlacidDreamer: Advancing Harmony in Text-to-3D Generation