Abstract:We introduce DreamCraft3D++, an extension of DreamCraft3D that enables efficient high-quality generation of complex 3D assets. DreamCraft3D++ inherits the multi-stage generation process of DreamCraft3D, but replaces the time-consuming geometry sculpting optimization with a feed-forward multi-plane based reconstruction model, speeding up the process by 1000x. For texture refinement, we propose a training-free IP-Adapter module that is conditioned on the enhanced multi-view images to enhance texture and geometry consistency, providing a 4x faster alternative to DreamCraft3D's DreamBooth fine-tuning. Experiments on diverse datasets demonstrate DreamCraft3D++'s ability to generate creative 3D assets with intricate geometry and realistic 360° textures, outperforming state-of-the-art image-to-3D methods in quality and speed. The full implementation will be open-sourced to enable new possibilities in 3D content creation.

What problem does this paper attempt to address?

The problem that this paper attempts to solve is to significantly improve the efficiency of complex 3D asset generation while maintaining high - quality generation. Specifically, the paper proposes a system named DreamCraft3D++, aiming to overcome several key challenges in existing 3D generation methods: 1. **Slow generation speed**: Existing 3D generation methods, such as DreamCraft3D, although able to generate high - quality 3D assets, have a long processing time and usually take several hours to complete the generation of a single object. This is unacceptable in practical applications, especially in scenarios that require rapid iteration and large - scale generation. 2. **Poor geometric and texture consistency**: Many optimization - based methods, when generating complex 3D content, although each view seems reasonable, lack consistency and coherence as a whole, resulting in the generated 3D models looking unnatural in the global view. 3. **Data scarcity**: 3D data is relatively scarce compared to 2D data, which limits the generalization ability of models trained on 3D data. Therefore, how to generate high - quality 3D assets based on limited 3D data is an important issue. To solve the above problems, DreamCraft3D++ introduces the following innovations: - **Multi - Plane Reconstruction Model (MP - LRM)**: Using a feed - forward multi - plane reconstruction model instead of the original time - consuming geometric carving optimization process, significantly improving the generation speed, achieving a 1,000 - fold acceleration effect. - **Untrained IP - Adapter module**: In order to enhance the consistency of texture and geometry, the paper proposes an untrained IP - Adapter module. This module enhances the consistency of texture and geometry through conditional multi - view images, providing an alternative that is 4 times faster than DreamCraft3D's DreamBooth fine - tuning. - **Efficient joint optimization of texture and geometry**: By dynamically selecting image embeddings based on camera positions, the IP - Adapter module can efficiently optimize texture and geometric details while maintaining consistency with multi - view input images. These innovations make DreamCraft3D++ not only significantly improve in generation speed, but also exceed existing 3D generation methods in generation quality, especially in dealing with complex geometric structures and realistic textures. The paper verifies these advantages through experiments on multiple datasets, demonstrating the great potential of DreamCraft3D++ in 3D content creation.

DreamCraft3D++: Efficient Hierarchical 3D Generation with Multi-Plane Reconstruction Model

DreamCraft3D: Hierarchical 3D Generation with Bootstrapped Diffusion Prior

GraphicsDreamer: Image to 3D Generation with Physical Consistency

MetaDreamer: Efficient Text-to-3D Creation With Disentangling Geometry and Texture

BoostDream: Efficient Refining for High-Quality Text-to-3D Generation from Multi-View Diffusion

DreamMesh: Jointly Manipulating and Texturing Triangle Meshes for Text-to-3D Generation

FlexiDreamer: Single Image-to-3D Generation with FlexiCubes

EfficientDreamer: High-Fidelity and Robust 3D Creation via Orthogonal-view Diffusion Prior

Rapid 3D Model Generation with Intuitive 3D Input

DreamControl: Control-Based Text-to-3D Generation with 3D Self-Prior

CraftsMan: High-fidelity Mesh Generation with 3D Native Generation and Interactive Geometry Refiner

Magic3D: High-Resolution Text-to-3D Content Creation

High-Fidelity 3D Model Generation with Relightable Appearance from Single Freehand Sketches and Text Guidance

DreamWorld: CUDA-accelerated Real-Time 3D Modeling System

HyperDreamer: Hyper-Realistic 3D Content Generation and Editing from a Single Image

ImageDream: Image-Prompt Multi-view Diffusion for 3D Generation

MicroDreamer: Efficient 3D Generation in $\sim$20 Seconds by Score-based Iterative Reconstruction

EucliDreamer: Fast and High-Quality Texturing for 3D Models with Depth-Conditioned Stable Diffusion

MicroDreamer: Efficient 3D Generation in ∼20 Seconds by Score-based Iterative Reconstruction

AnimatableDreamer: Text-Guided Non-rigid 3D Model Generation and Reconstruction with Canonical Score Distillation

X-Dreamer: Creating High-quality 3D Content by Bridging the Domain Gap Between Text-to-2D and Text-to-3D Generation