Abstract:We revisit the work of Mitter and Newton on an information-theoretic interpretation of Bayes' formula through the Gibbs variational principle. This formulation allowed them to pose nonlinear estimation for diffusion processes as a problem in stochastic optimal control, so that the posterior density of the signal given the observation path could be sampled by adding a drift to the signal process. We show that this control-theoretic approach to sampling provides a common mechanism underlying several distinct problems involving diffusion processes, specifically importance sampling using Feynman-Kac averages, time reversal, and Schrödinger bridges.
What problem does this paper attempt to address?
The problem that this paper attempts to solve is sampling during the diffusion process via variational methods. Specifically, the author re - examines the work of Mitter and Newton, who provided an information - theoretic interpretation of Bayes' formula through the Gibbs variational principle. This interpretation enables the transformation of nonlinear estimation problems into stochastic optimal control problems, so that samples can be obtained from the posterior density by adding a drift to the signal process. This paper shows that this control - theory - based method of free - energy minimization provides a unified mechanism applicable to several different problems involving diffusion processes, including importance sampling using Feynman - Kac averaging, time - reversal, and the Schrödinger bridge problem.
### Main contributions of the paper
1. **Unified perspective**: This paper provides a unified perspective, explaining that the specific constructions that emerge when solving these problems can all be regarded as instances of stochastic optimal control of diffusion processes.
2. **Application of variational methods**: Through variational methods, the author shows how to reduce these different problems to free - energy minimization problems, thus providing a systematic method for dealing with these problems.
3. **Solutions to specific problems**:
- **Importance sampling**: Sampling is carried out via Feynman - Kac averaging.
- **Time - reversal**: The time - reversal problem of the diffusion process is solved.
- **Schrödinger bridge problem**: The Schrödinger bridge problem is solved through the optimal control method.
### Mathematical background
The mathematical tools involved in the paper include:
- **Gibbs variational principle**: Used to minimize the free - energy functional.
- **Relative entropy**: Defined as \( D(\tilde{P} \| P) = \int_X \log\left(\frac{d\tilde{P}}{dP}\right) d\tilde{P} \).
- **Free energy**: Defined as \( F(\tilde{P}) = \langle H, \tilde{P} \rangle + D(\tilde{P} \| P) \).
- **Hamilton - Jacobi - Bellman equation**: Used to describe the value function of the optimal control problem.
### Mathematical formulations of specific problems
1. **Feynman - Kac averaging**:
- The objective is to generate samples from the Gibbs measure \( P^* \), where \( P^* \) has the form:
\[
\frac{dP^*}{dP} = \frac{\exp(-H)}{\int_X \exp(-H) dP}
\]
- This problem can be transformed into calculating or estimating the Feynman - Kac average:
\[
\langle F, P^* \rangle = \frac{\int_X F \exp(-H) dP}{\int_X \exp(-H) dP}
\]
2. **Schrödinger bridge problem**:
- Given two Borel probability measures \( \mu \) and \( \mu' \), the objective is to find an acceptable drift \( u \), such that the energy \( \frac{1}{2} \tilde{E} \int_0^T |\sigma^T u(\tilde{X}_t, t)|^2 dt \) is minimized while satisfying \( \tilde{X}_T \sim \mu' \).
- Through the free - energy minimization method, the corresponding Gibbs measure on the path space can be explicitly identified.
3. **Time - reversal**:
- Consider an \( n \)-dimensional diffusion process \( X_t \), whether its time - reversal \( \bar{X}_t = X_{T - t} \) is also a diffusion process.
- Through the optimal control method, the drift and diffusion coefficients of the time - reversal process can be derived.
### Conclusion
This paper provides a unified framework through variational methods and control theory to solve problems involving diffusion.