Abstract:Recent advances in text-to-3D generation have made significant progress. In particular, with the pretrained diffusion models, existing methods predominantly use Score Distillation Sampling (SDS) to train 3D models such as Neural RaRecent advances in text-to-3D generation have made significant progress. In particular, with the pretrained diffusion models, existing methods predominantly use Score Distillation Sampling (SDS) to train 3D models such as Neural Radiance Fields (NeRF) and 3D Gaussian Splatting (3D GS). However, a hurdle is that they often encounter difficulties with over-smoothing textures and over-saturating colors. The rectified flow model -- which utilizes a simple ordinary differential equation (ODE) to represent a straight trajectory -- shows promise as an alternative prior to text-to-3D generation. It learns a time-independent vector field, thereby reducing the ambiguity in 3D model update gradients that are calculated using time-dependent scores in the SDS framework. In light of this, we first develop a mathematical analysis to seamlessly integrate SDS with rectified flow model, paving the way for our initial framework known as Vector Field Distillation Sampling (VFDS). However, empirical findings indicate that VFDS still results in over-smoothing outcomes. Therefore, we analyze the grounding reasons for such a failure from the perspective of ODE trajectories. On top, we propose a novel framework, named FlowDreamer, which yields high fidelity results with richer textual details and faster convergence. The key insight is to leverage the coupling and reversible properties of the rectified flow model to search for the corresponding noise, rather than using randomly sampled noise as in VFDS. Accordingly, we introduce a novel Unique Couple Matching (UCM) loss, which guides the 3D model to optimize along the same trajectory.

Sequential Flow Straightening for Generative Modeling

Exploring Straighter Trajectories of Flow Matching with Diffusion Guidance

2-Rectifications are Enough for Straight Flows: A Theoretical Insight into Wasserstein Convergence

FlowDreamer: Exploring High Fidelity Text-to-3D Generation via Rectified Flow

Constant Acceleration Flow

Simple ReFlow: Improved Techniques for Fast Flow Models

Diffusion Generative Flow Samplers: Improving learning signals through partial trajectory optimization

Improving the Training of Rectified Flows

Generative Modeling with Flow-Guided Density Ratio Learning

Switched Flow Matching: Eliminating Singularities via Switching ODEs

Fisher Flow Matching for Generative Modeling over Discrete Data

Language Rectified Flow: Advancing Diffusion Language Generation with Probabilistic Flows

Rectified Diffusion: Straightness Is Not Your Need in Rectified Flow

SlimFlow: Training Smaller One-Step Diffusion Models with Rectified Flow

Local Flow Matching Generative Models

Steering Rectified Flow Models in the Vector Field for Controlled Image Generation

D-Flow: Differentiating through Flows for Controlled Generation

FlowTurbo: Towards Real-time Flow-Based Image Generation with Velocity Refiner

A Sharp Convergence Theory for The Probability Flow ODEs of Diffusion Models

Stochastic Sampling from Deterministic Flow Models

Annealing Flow Generative Model Towards Sampling High-Dimensional and Multi-Modal Distributions