Abstract:Training-free conditional generation aims to leverage the unconditional diffusion models to implement the conditional generation, where flow-matching (FM) and diffusion probabilistic models (DPMs) are two mature unconditional diffusion models that achieve high-quality generation. Two questions were asked in this paper: What are the underlying connections between FM and DPMs in training-free conditional generation? Can we leverage DPMs to improve the training-free conditional generation for FM? We first show that a probabilistic diffusion path can be associated with the FM and DPMs. Then, we reformulate the ordinary differential equation (ODE) of FM based on the score function of DPMs, and thus, the conditions in FM can be incorporated as those in DPMs. Finally, we propose two posterior sampling methods to estimate the conditional term and achieve a training-free conditional generation of FM. Experimental results show that our proposed method could be implemented for various conditional generation tasks. Our method can generate higher-quality results than the state-of-the-art methods.
What problem does this paper attempt to address?
### Problems the Paper Aims to Solve
This paper aims to address two key issues in training-free conditional generation:
1. **What is the intrinsic connection between Flow Matching (FM) and Diffusion Probabilistic Models (DPMs)?**
- The paper explores how to utilize DPMs to improve the conditional generation capability of FM under training-free conditions. Specifically, the authors attempt to reveal whether there is a probabilistic path connection between FM and DPMs.
2. **Can DPMs be used to improve FM's training-free conditional generation?**
- The authors propose a new method called Flow-Matching based Posterior Sampling (FMPS), which applies posterior sampling techniques from DPMs to FM to achieve training-free conditional generation.
### Background and Motivation
- **Training-free conditional generation**: The goal of this technique is to generate images that meet specific conditions using pre-trained unconditional diffusion models (such as DPMs and FMs) without retraining the model.
- **Diffusion Probabilistic Models (DPMs)**: These have been widely used for high-quality image generation, and their posterior sampling methods perform well under training-free conditions.
- **Flow Matching (FMs)**: Although promising, there has been little research on training-free conditional generation, and there is a lack of direct methods to introduce conditions.
### Main Contributions
1. **Proposed FMPS**: A new training-free conditional generation method suitable for flow diffusion models (FMs).
2. **Revealed the connection between FM and DPMs**: By redefining the ordinary differential equation (ODE) of FMs, making it possible to explicitly represent the score function, thereby introducing conditions.
3. **Experimental results**: Demonstrated the superior performance of FMPS in various downstream tasks (such as linear inverse problems, nonlinear inverse problems, and text-to-image generation), verifying its effectiveness and efficiency.
### Method Overview
- **Redefining the ODE of FMs**: By introducing the score function, the reverse process of FMs is reformulated to incorporate conditions.
- **Posterior sampling**: Utilizing posterior sampling techniques from DPMs to achieve training-free conditional generation by estimating the conditional term.
- **Two posterior sampling methods**: FMPS-gradient (gradient-aware) and FMPS-free (gradient-free), suitable for different scenarios.
### Experimental Results
- **Linear inverse problems**: On the CelebA-HQ dataset, FMPS performed excellently in tasks such as image inpainting, super-resolution, and Gaussian deblurring, outperforming existing methods.
- **Nonlinear inverse problems**: FMPS also achieved excellent generation quality in tasks such as segmentation maps, sketches, and facial recognition.
- **Text style generation**: FMPS generated high-quality images under complex text prompts, maintaining style consistency.
### Conclusion
The paper proposes a new training-free conditional generation method, FMPS, by revealing the connection between FM and DPMs. It successfully applies posterior sampling techniques from DPMs to FMs, achieving high-quality conditional generation. Experimental results validate the effectiveness and efficiency of FMPS.