Abstract:We propose a systematic training-free method to transform the probability flow of a "linear" stochastic process characterized by the equation X_{t}=a_{t}X_{0}+\sigma_{t}X_{1} into a straight constant-speed (SC) flow, reminiscent of Rectified Flow. This transformation facilitates fast sampling along the original probability flow via the Euler method without training a new model of the SC flow. The flexibility of our approach allows us to extend our transformation to inter-convert two posterior flows of two distinct linear stochastic processes. Moreover, we can easily integrate high-order numerical solvers into the transformed SC flow, further enhancing the sampling accuracy and efficiency. Rigorous theoretical analysis and extensive experimental results substantiate the advantages of our framework. Our code is available at this [<a class="link-external link-https" href="https://github.com/clarken92/VFM" rel="external noopener nofollow">this https URL</a>||link].
What problem does this paper attempt to address?
The main problem that this paper attempts to solve is how to convert complex, non - linear probability flows (posterior flow) into straight and constant - speed probability flows (straight constant - speed, SC flow), thereby accelerating the sampling process and improving efficiency. Specifically, the paper proposes a systematic training - free method that can convert the probability flow of any linear stochastic process into an SC flow similar to Rectified Flow. This conversion makes it possible to perform rapid sampling on the original probability flow by the Euler method without retraining a new model.
### Core contributions of the paper
1. **Training - free conversion method**: The paper proposes a method that does not require additional training and can directly convert complex probability flows into straight and constant - speed probability flows. This greatly reduces the demand for computational resources, especially for large - scale models such as Stable Diffusion.
2. **Flexibility and generality**: This method is not only applicable to the conversion of a single probability flow, but can also be extended to the mutual conversion of posterior flows between two different linear stochastic processes. In addition, this method is applicable to all existing diffusion models, including DDPM/VP SDE and SMLD/VE SDE, etc.
3. **Integration of high - order numerical solvers**: The paper also explores how to seamlessly integrate high - order numerical ODE solvers (such as Runge - Kutta and linear multi - step methods) into the converted SC flow to further improve sampling accuracy and efficiency. These solvers can significantly reduce the number of sampling steps, making the results comparable to the current state - of - the - art training - free samplers.
### Problems solved
- **Low sampling efficiency**: Traditional diffusion models usually require hundreds to thousands of function evaluations to generate high - quality images because of the random noise in each SDE update step. By converting to SC flow, sampling can be carried out using statistically equivalent ODEs, thereby significantly reducing the required sampling steps.
- **High resource consumption**: Existing methods (such as training from scratch or distilling Rectified Flow) require a large amount of resources, especially for large - scale models. The conversion method proposed in the paper avoids the need for retraining, thereby saving a large amount of computational resources.
- **Lack of flexibility**: Existing methods lack flexibility when converting other posterior flows and need to retrain new models. The method proposed in the paper can achieve flexible conversion through simple variable transformation.
### Method overview
The paper converts non - linear flows into linear flows by introducing techniques such as variable scaling and time adjustment, and then further converts them into straight flows with constant speed. The specific steps include:
1. **Variable scaling**: By scaling the variables in the original stochastic process, convert it into a linear flow with a simpler form.
2. **Time adjustment**: By changing the time parameter, convert the linear flow into a straight flow with constant speed.
3. **High - order numerical solver**: Apply a high - order numerical solver on the converted SC flow to improve sampling accuracy and efficiency.
### Conclusion
The method proposed in the paper provides an efficient and flexible solution that can significantly improve the sampling efficiency of diffusion models without increasing the training cost. Through theoretical analysis and experimental verification, the effectiveness and superiority of this method are proved.