Abstract:Generating diverse and high-quality 3D assets automatically poses a fundamental yet challenging task in 3D computer vision. Despite extensive efforts in 3D generation, existing optimization-based approaches struggle to produce large-scale 3D assets efficiently. Meanwhile, feed-forward methods often focus on generating only a single category or a few categories, limiting their generalizability. Therefore, we introduce a diffusion-based feed-forward framework to address these challenges with a single model. To handle the large diversity and complexity in geometry and texture across categories efficiently, we 1) adopt improved triplane to guarantee efficiency; 2) introduce the 3D-aware transformer to aggregate the generalized 3D knowledge with specialized 3D features; and 3) devise the 3D-aware encoder/decoder to enhance the generalized 3D knowledge. Building upon our 3D-aware Diffusion model with TransFormer, DiffTF, we propose a stronger version for 3D generation, i.e., DiffTF++. It boils down to two parts: multi-view reconstruction loss and triplane refinement. Specifically, we utilize multi-view reconstruction loss to fine-tune the diffusion model and triplane decoder, thereby avoiding the negative influence caused by reconstruction errors and improving texture synthesis. By eliminating the mismatch between the two stages, the generative performance is enhanced, especially in texture. Additionally, a 3D-aware refinement process is introduced to filter out artifacts and refine triplanes, resulting in the generation of more intricate and reasonable details. Extensive experiments on ShapeNet and OmniObject3D convincingly demonstrate the effectiveness of our proposed modules and the state-of-the-art 3D object generation performance with large diversity, rich semantics, and high quality.

DNF: Unconditional 4D Generation with Dictionary-based Neural Fields

4Diffusion: Multi-view Video Diffusion Model for 4D Generation

Neural 4D Evolution under Large Topological Changes from 2D Images

Beyond Skeletons: Integrative Latent Mapping for Coherent 4D Sequence Generation

Part-aware Shape Generation with Latent 3D Diffusion of Neural Voxel Fields

LN3Diff: Scalable Latent Neural Fields Diffusion for Speedy 3D Generation

4Dynamic: Text-to-4D Generation with Hybrid Priors

Diffusion-SDF: Conditional Generative Modeling of Signed Distance Functions

4D Facial Expression Diffusion Model

Human4DiT: 360-degree Human Video Generation with 4D Diffusion Transformer

SLS4D: Sparse Latent Space for 4D Novel View Synthesis

Animate124: Animating One Image to 4D Dynamic Scene

SC4D: Sparse-Controlled Video-to-4D Generation and Motion Transfer

Locally Attentional SDF Diffusion for Controllable 3D Shape Generation

Comp4D: LLM-Guided Compositional 4D Scene Generation

TC4D: Trajectory-Conditioned Text-to-4D Generation

H4D: Human 4D Modeling by Learning Neural Compositional Representation

EG4D: Explicit Generation of 4D Object without Score Distillation

STAG4D: Spatial-Temporal Anchored Generative 4D Gaussians

NeuSDFusion: A Spatial-Aware Generative Model for 3D Shape Completion, Reconstruction, and Generation

DiffTF++: 3D-aware Diffusion Transformer for Large-Vocabulary 3D Generation