Abstract:Encouraged by the growing availability of pre-trained 2D diffusion models, image-to-3D generation by leveraging Score Distillation Sampling (SDS) is making remarkable progress. Most existing methods combine novel-view lifting from 2D diffusion models which usually take the reference image as a condition while applying hard L2 image supervision at the reference view. Yet heavily adhering to the image is prone to corrupting the inductive knowledge of the 2D diffusion model leading to flat or distorted 3D generation frequently. In this work, we reexamine image-to-3D in a novel perspective and present Isotropic3D, an image-to-3D generation pipeline that takes only an image CLIP embedding as input. Isotropic3D allows the optimization to be isotropic w.r.t. the azimuth angle by solely resting on the SDS loss. The core of our framework lies in a two-stage diffusion model fine-tuning. Firstly, we fine-tune a text-to-3D diffusion model by substituting its text encoder with an image encoder, by which the model preliminarily acquires image-to-image capabilities. Secondly, we perform fine-tuning using our Explicit Multi-view Attention (EMA) which combines noisy multi-view images with the noise-free reference image as an explicit condition. CLIP embedding is sent to the diffusion model throughout the whole process while reference images are discarded once after fine-tuning. As a result, with a single image CLIP embedding, Isotropic3D is capable of generating multi-view mutually consistent images and also a 3D model with more symmetrical and neat content, well-proportioned geometry, rich colored texture, and less distortion compared with existing image-to-3D methods while still preserving the similarity to the reference image to a large extent. The project page is available at

Hybrid Fourier Score Distillation for Efficient One Image to 3D Object Generation

Enhancing Single Image to 3D Generation using Gaussian Splatting and Hybrid Diffusion Priors

Stable Score Distillation for High-Quality 3D Generation

EG4D: Explicit Generation of 4D Object without Score Distillation

HiFi-123: Towards High-fidelity One Image to 3D Content Generation

Novel 3D-Aware Composition Images Synthesis for Object Display with Diffusion Model.

Magic123: One Image to High-Quality 3D Object Generation Using Both 2D and 3D Diffusion Priors

Score Distillation via Reparametrized DDIM

Connecting Consistency Distillation to Score Distillation for Text-to-3D Generation

Baking Gaussian Splatting into Diffusion Denoiser for Fast and Scalable Single-stage Image-to-3D Generation

Isotropic3D: Image-to-3D Generation Based on a Single CLIP Embedding

Efficient Text-Guided 3D-Aware Portrait Generation with Score Distillation Sampling on Distribution

DreamTime: An Improved Optimization Strategy for Diffusion-Guided 3D Generation

Diffusion Time-step Curriculum for One Image to 3D Generation

Learn to Optimize Denoising Scores: A Unified and Improved Diffusion Prior for 3D Generation

One-2-3-45++: Fast Single Image to 3D Objects with Consistent Multi-View Generation and 3D Diffusion

AGG: Amortized Generative 3D Gaussians for Single Image to 3D

Text-Guided 3D Object Generation Via Disentangled Shape and Appearance Score Distillation Sampling

Direct2.5: Diverse Text-to-3D Generation via Multi-view 2.5D Diffusion

VividDreamer: Invariant Score Distillation For Hyper-Realistic Text-to-3D Generation