Dogyun Park,Sojin Lee,Sihyeon Kim,Taehoon Lee,Youngjoon Hong,Hyunwoo J. Kim
Abstract:Rectified flow and reflow procedures have significantly advanced fast generation by progressively straightening ordinary differential equation (ODE) flows. They operate under the assumption that image and noise pairs, known as couplings, can be approximated by straight trajectories with constant velocity. However, we observe that modeling with constant velocity and using reflow procedures have limitations in accurately learning straight trajectories between pairs, resulting in suboptimal performance in few-step generation. To address these limitations, we introduce Constant Acceleration Flow (CAF), a novel framework based on a simple constant acceleration equation. CAF introduces acceleration as an additional learnable variable, allowing for more expressive and accurate estimation of the ODE flow. Moreover, we propose two techniques to further improve estimation accuracy: initial velocity conditioning for the acceleration model and a reflow process for the initial velocity. Our comprehensive studies on toy datasets, CIFAR-10, and ImageNet 64x64 demonstrate that CAF outperforms state-of-the-art baselines for one-step generation. We also show that CAF dramatically improves few-step coupling preservation and inversion over Rectified flow. Code is available at \href{<a class="link-external link-https" href="https://github.com/mlvlab/CAF" rel="external noopener nofollow">this https URL</a>}{<a class="link-external link-https" href="https://github.com/mlvlab/CAF" rel="external noopener nofollow">this https URL</a>}.
What problem does this paper attempt to address?
### Problems the Paper Attempts to Solve
This paper attempts to address several key issues in generative models, particularly those based on the Rectified Flow method:
1. **Limitations of Constant Velocity Modeling**: Existing rectified flow methods assume that the trajectory between images and noise pairs can be approximated by a straight-line trajectory with constant speed. However, this assumption has limitations in practical applications, leading to poor generation performance, especially in few-step generation tasks.
2. **Flow Crossing Problem**: During training, due to the random pairing of images and noise pairs, the trajectories in rectified flow methods may cross (flow crossing), causing the sampling trajectories to become curved, which affects the model's learning effectiveness and generation quality.
3. **Balance Between Generation Efficiency and Quality**: Existing generative models typically require a multi-step generation process, which is not only slow but also computationally heavy. How to improve generation speed while maintaining generation quality is an important research direction.
To address these issues, the paper proposes a new framework—**Constant Acceleration Flow (CAF)**. CAF improves the expressiveness and accuracy of the Ordinary Differential Equation (ODE) flow by introducing acceleration as an additional learnable variable. Additionally, the paper proposes two techniques to further improve estimation accuracy: Initial Velocity Conditioning (IVC) and a reflow process for initial velocity. These techniques help reduce the flow crossing problem and improve generation quality and efficiency.
### Main Contributions
1. **Proposing Constant Acceleration Flow (CAF)**: This is a new ODE framework that enhances the estimation accuracy of ODE flows by introducing acceleration as a controllable variable, offering higher expressiveness compared to constant velocity models.
2. **Solving the Flow Crossing Problem**: The paper proposes two strategies, Initial Velocity Conditioning (IVC) and the reflow process, to ensure more accurate trajectory estimation even in the presence of flow crossing.
3. **Experimental Validation**: Extensive experiments on synthetic datasets, CIFAR-10, and ImageNet 64×64 demonstrate that CAF outperforms existing rectified flow methods and other baseline models in terms of generation quality, coupling retention, straightness, and reverse generation.
### Experimental Results
- **CIFAR-10**: In the unconditional generation task, CAF achieved an FID score of 4.81, significantly outperforming other methods. In the conditional generation task, CAF achieved an FID score of 1.39, surpassing recent strong baseline methods.
- **ImageNet 64×64**: In the one-step generation task, CAF achieved an FID score of 1.69, showing excellent performance and surpassing other strong baseline methods.
### Conclusion
By introducing the Constant Acceleration Flow (CAF) framework, the paper effectively addresses the limitations of rectified flow methods in generative models, particularly in few-step generation tasks. CAF significantly improves generation quality and efficiency by introducing acceleration and two techniques (IVC and the reflow process), providing new ideas and methods for generative model research.