Abstract:Diffusion models have emerged as a powerful tool for image generation and denoising. Typically, generative models learn a trajectory between the starting noise distribution and the target data distribution. Recently Liu et al. (2023b) designed a novel alternative generative model Rectified Flow (RF), which aims to learn straight flow trajectories from noise to data using a sequence of convex optimization problems with close ties to optimal transport. If the trajectory is curved, one must use many Euler discretization steps or novel strategies, such as exponential integrators, to achieve a satisfactory generation quality. In contrast, RF has been shown to theoretically straighten the trajectory through successive rectifications, reducing the number of function evaluations (NFEs) while sampling. It has also been shown empirically that RF may improve the straightness in two rectifications if one can solve the underlying optimization problem within a sufficiently small error. In this paper, we make two key theoretical contributions: 1) we provide the first theoretical analysis of the Wasserstein distance between the sampling distribution of RF and the target distribution. Our error rate is characterized by the number of discretization steps and a \textit{new formulation of straightness} stronger than that in the original work. 2) under a mild regularity assumption, we show that for a rectified flow from a Gaussian to any general target distribution with finite first moment (e.g. mixture of Gaussians), two rectifications are sufficient to achieve a straight flow, which is in line with the previous empirical findings. Additionally, we also present empirical results on both simulated and real datasets to validate our theoretical findings.

What problem does this paper attempt to address?

The main problem that this paper attempts to solve is about the theoretical understanding of Rectified Flow (RF) in generative models and its convergence under the Wasserstein distance. Specifically, the paper focuses on the following two aspects of problems: 1. **Influence of Wasserstein Convergence and Straightness**: - It provides the first theoretical analysis of the 2 - Wasserstein distance between the RF sampling distribution and the target distribution. - It analyzes the influence of the straightness parameter on the convergence speed. In particular, when the flow is close to a straight line, the number of discretization steps can be significantly reduced, thus improving the sampling efficiency. 2. **Whether Two Rectifications Are Sufficient to Achieve a Straight Flow**: - Under mild regularity conditions, it is proved that from a Gaussian distribution to any target distribution with finite first - order moments (such as a Gaussian mixture distribution), a straight flow can be achieved with only two rectifications. - This result provides a theoretical basis for previous experimental findings, that is, usually only two rectifications are required to produce a flow close to a straight line. ### Specific Problems and Contributions #### 1. Wasserstein Convergence - **Theoretical Analysis**: The author establishes a new upper bound of the squared 2 - Wasserstein distance, which mainly depends on the estimation error of the velocity function and the discretization error caused by the Euler discretization scheme. - **Straightness Parameter**: New straightness parameters are introduced. These parameters take small values for approximately straight flows, explaining why sampling can be carried out with fewer discretization steps under approximately straight flows. #### 2. Sufficient Conditions for Straightness - **Straightness with Two Rectifications**: For the first time, it is theoretically proved that under mild regularity conditions, from a standard Gaussian distribution to a general target distribution (including Gaussian mixture distributions), a straight flow can be achieved with only two rectifications. - **Geometric Intuition**: The flow geometric properties when the source distribution and the target distribution are respectively Gaussian and simple Gaussian mixture distributions are studied, further verifying the sufficiency of two rectifications. ### Paper Structure - **Section 2**: Introduces the optimal transport theory and its relationship with rectified flows. - **Section 3**: Presents the main convergence results of continuous - time and discretized rectified flows under the 2 - Wasserstein metric and introduces new straightness parameters. - **Section 4**: Establishes the general straightness results of 2 - RF and establishes the geometric intuition of rectified flows through simple examples. - **Section 5**: Verifies the theoretical results through simulation and real - data experiments. ### Formula Representation - **Wasserstein Distance**: \[ W_2(\rho_1, \rho_2) \] - **Average Straightness Parameter**: \[ \gamma_1(Z) = \int_0^1 \mathbb{E}[\|\dot{v}_t(Z_t)\|_2^2] \, dt \] - **Piecewise Straightness Parameter**: \[ \gamma_{2,T}(Z) = \max_{i \in [T]} \frac{1}{t_i - t_{i - 1}} \int_{t_{i - 1}}^{t_i} \mathbb{E}[\|\dot{v}_t(Z_t)\|_2^2] \, dt \] Through these theoretical analyses and empirical studies, the paper provides a solid theoretical foundation for rectified flows in generative models and shows their potential in practical applications.

2-Rectifications are Enough for Straight Flows: A Theoretical Insight into Wasserstein Convergence

Rectified Diffusion: Straightness Is Not Your Need in Rectified Flow

Improving the Training of Rectified Flows

Sequential Flow Straightening for Generative Modeling

Steering Rectified Flow Models in the Vector Field for Controlled Image Generation

Text-to-Image Rectified Flow as Plug-and-Play Priors

A Sharp Convergence Theory for The Probability Flow ODEs of Diffusion Models

FlowDreamer: Exploring High Fidelity Text-to-3D Generation via Rectified Flow

Iterated Schrödinger bridge approximation to Wasserstein Gradient Flows

Convergence Analysis for General Probability Flow ODEs of Diffusion Models in Wasserstein Distances

Semantic Image Inversion and Editing using Rectified Stochastic Differential Equations

Convergence of flow-based generative models via proximal gradient descent in Wasserstein space

Scaling Rectified Flow Transformers for High-Resolution Image Synthesis

Diffusion models for Gaussian distributions: Exact solutions and Wasserstein errors

An Optimal Transport Analogue of the Rudin–Osher–Fatemi Model and Its Corresponding Multiscale Theory

Gaussian Interpolation Flows

Exploring Straighter Trajectories of Flow Matching with Diffusion Guidance

Gradient Flows and Riemannian Structure in the Gromov-Wasserstein Geometry

Bridging the Gap Between Variational Inference and Wasserstein Gradient Flows

An optimal transport analogue of the Rudin Osher Fatemi model and its corresponding multiscale theory

Stochastic Modified Flows for Riemannian Stochastic Gradient Descent