Abstract:Generating diverse and high-quality 3D assets automatically poses a fundamental yet challenging task in 3D computer vision. Despite extensive efforts in 3D generation, existing optimization-based approaches struggle to produce large-scale 3D assets efficiently. Meanwhile, feed-forward methods often focus on generating only a single category or a few categories, limiting their generalizability. Therefore, we introduce a diffusion-based feed-forward framework to address these challenges with a single model. To handle the large diversity and complexity in geometry and texture across categories efficiently, we 1) adopt improved triplane to guarantee efficiency; 2) introduce the 3D-aware transformer to aggregate the generalized 3D knowledge with specialized 3D features; and 3) devise the 3D-aware encoder/decoder to enhance the generalized 3D knowledge. Building upon our 3D-aware Diffusion model with TransFormer, DiffTF, we propose a stronger version for 3D generation, i.e., DiffTF++. It boils down to two parts: multi-view reconstruction loss and triplane refinement. Specifically, we utilize multi-view reconstruction loss to fine-tune the diffusion model and triplane decoder, thereby avoiding the negative influence caused by reconstruction errors and improving texture synthesis. By eliminating the mismatch between the two stages, the generative performance is enhanced, especially in texture. Additionally, a 3D-aware refinement process is introduced to filter out artifacts and refine triplanes, resulting in the generation of more intricate and reasonable details. Extensive experiments on ShapeNet and OmniObject3D convincingly demonstrate the effectiveness of our proposed modules and the state-of-the-art 3D object generation performance with large diversity, rich semantics, and high quality.

3DTopia-XL: Scaling High-quality 3D Asset Generation via Primitive Diffusion

3dtopia-xl: Scaling high-quality 3d asset generation via primitive diffusion

3dtopia: Large text-to-3d generation model with hybrid diffusion priors

DiffTF++: 3D-aware Diffusion Transformer for Large-Vocabulary 3D Generation

Direct3D: Scalable Image-to-3D Generation via 3D Latent Diffusion Transformer

BoostDream: Efficient Refining for High-Quality Text-to-3D Generation from Multi-View Diffusion

Novel 3D-Aware Composition Images Synthesis for Object Display with Diffusion Model.

GET3D: A Generative Model of High Quality 3D Textured Shapes Learned from Images

Direct2.5: Diverse Text-to-3D Generation via Multi-view 2.5D Diffusion

Tencent Hunyuan3D-1.0: A Unified Framework for Text-to-3D and Image-to-3D Generation

HexaGen3D: StableDiffusion is just one step away from Fast and Diverse Text-to-3D Generation

ScalingGaussian: Enhancing 3D Content Creation with Generative Gaussian Splatting

EfficientDreamer: High-Fidelity and Robust 3D Creation via Orthogonal-view Diffusion Prior

PrimDiffusion: Volumetric Primitives Diffusion for 3D Human Generation

One-2-3-45++: Fast Single Image to 3D Objects with Consistent Multi-View Generation and 3D Diffusion

Edify 3D: Scalable High-Quality 3D Asset Generation

GraphicsDreamer: Image to 3D Generation with Physical Consistency

Unique3D: High-Quality and Efficient 3D Mesh Generation from a Single Image