Abstract:Due to their uncertainty quantification, Bayesian solutions to inverse problems are the framework of choice in applications that are risk averse. These benefits come at the cost of computations that are in general, intractable. New advances in machine learning and variational inference (VI) have lowered the computational barrier by learning from examples. Two VI paradigms have emerged that represent different tradeoffs: amortized and non-amortized. Amortized VI can produce fast results but due to generalizing to many observed datasets it produces suboptimal inference results. Non-amortized VI is slower at inference but finds better posterior approximations since it is specialized towards a single observed dataset. Current amortized VI techniques run into a sub-optimality wall that can not be improved without more expressive neural networks or extra training data. We present a solution that enables iterative improvement of amortized posteriors that uses the same networks architectures and training data. The benefits of our method requires extra computations but these remain frugal since they are based on physics-hybrid methods and summary statistics. Importantly, these computations remain mostly offline thus our method maintains cheap and reusable online evaluation while bridging the approximation gap these two paradigms. We denote our proposed method ASPIRE - Amortized posteriors with Summaries that are Physics-based and Iteratively REfined. We first validate our method on a stylized problem with a known posterior then demonstrate its practical use on a high-dimensional and nonlinear transcranial medical imaging problem with ultrasound. Compared with the baseline and previous methods from the literature our method stands out as an computationally efficient and high-fidelity method for posterior inference.

What problem does this paper attempt to address?

The problem that this paper attempts to solve is how to improve the quality of posterior inference while maintaining computational efficiency in Bayesian inverse problems. Specifically, the paper focuses on how to combine physical knowledge and iterative optimization techniques to improve the existing variational inference methods, especially for Bayesian inverse problems in high - dimensional, nonlinear medical imaging problems. ### Background and Challenges Bayesian inverse problems have advantages in dealing with uncertainty quantification, but in practical applications, they face the problem of high computational complexity. Traditional variational inference (VI) methods are divided into two categories: **Non - amortized VI** and **Amortized VI**. The former can provide more accurate posterior estimates, but has a high computational cost and is not suitable for rapid inference; the latter has high computational efficiency, but the quality of posterior estimates is low, and there is a so - called "amortization gap". ### Main Contributions of the Paper 1. **Propose the ASPIRE algorithm**: By iteratively optimizing the amortized posterior distribution and combining physics - based summary statistics, gradually improve the quality of posterior estimates while maintaining low online computational costs. 2. **Verify the effectiveness of the method**: Verify on idealized examples that the ASPIRE algorithm can accurately approximate the true posterior distribution. 3. **Apply to practical problems**: Evaluate the performance of the ASPIRE algorithm on high - dimensional, nonlinear transcranial ultrasound medical imaging problems, focusing on the accuracy of the posterior mean and the uncertainty of the prediction reconstruction error. 4. **Introduce a new non - amortized inference method**: As a "gold standard" for comparison. 5. **Quantitative and qualitative comparison**: Compare with the baseline methods in the existing literature and the "gold standard" non - amortized inference method to show the advantages of ASPIRE. 6. **Cost - benefit analysis**: Analyze the trade - off between the computational cost of offline training and the fast online capabilities of the amortized method. ### Method Overview 1. **Amortized variational inference and conditional normalizing flows**: Use conditional normalizing flows (CNFs) for amortized variational inference, and generate posterior samples by learning an invertible transformation from the data distribution to the standard normal distribution. 2. **Score - based summary statistics**: Use the log - likelihood gradient as summary statistics to capture the main features of the data and reduce the dimension. 3. **Iterative optimization**: Continuously optimize summary statistics and posterior estimates through an iterative process. In each iteration, use the current summary statistics to train the CNF, then generate new posterior samples, calculate the average of these samples as a new summary point, and then use these new summary points to calculate new summary statistics, and so on for multiple cycles. ### Key Formulas - **Bayesian formula**: \[ p(x|y) \propto p(y|x) p(x) \] - **Forward KL divergence**: \[ KL(p(x|y)\|p_{\theta}(x|y))=\mathbb{E}_{p(x|y)}[-\log p_{\theta}(x|y)+\log p(x|y)] \] - **Score - based summary statistics**: \[ y = \nabla_x\log p(y|x)\big|_{x_0}=\nabla F^{\top}[x_0](F(x_0)-y) \] ### Conclusion By combining physical knowledge and iterative optimization techniques, the ASPIRE algorithm can significantly improve the quality of posterior inference while maintaining computational efficiency, and is especially suitable for high - dimensional, nonlinear medical imaging problems.

ASPIRE: Iterative Amortized Posterior Inference for Bayesian Inverse Problems

Refining Amortized Posterior Approximations using Gradient-Based Summary Statistics

Bayesian Inference and Deep Learning for Inverse Problems

A Variational Bayesian Approximation Approach Via A Sparsity Enforcing Prior In Acoustic Imaging

Learning to solve Bayesian inverse problems: An amortized variational inference approach using Gaussian and Flow guides

A probabilistic approach to tomography and adjoint state methods, with an application to full waveform inversion in medical ultrasound

ADAVI: Automatic Dual Amortized Variational Inference Applied To Pyramidal Bayesian Models

Posterior Estimation Using Deep Learning: A Simulation Study of Compartmental Modeling in Dynamic PET

Posterior temperature optimized Bayesian models for inverse problems in medical imaging

Posterior estimation using deep learning: a simulation study of compartmental modeling in dynamic positron emission tomography

Diffusion Prior-Based Amortized Variational Inference for Noisy Inverse Problems

Efficient Bayesian Physics Informed Neural Networks for inverse problems via Ensemble Kalman Inversion

Efficient hyperparameter estimation in Bayesian inverse problems using sample average approximation

Variational Prior Replacement in Bayesian Inference and Inversion

VI-DGP: A Variational Inference Method with Deep Generative Prior for Solving High-Dimensional Inverse Problems

Amortized Bayesian Decision Making for simulation-based models

Residual-based error correction for neural operator accelerated infinite-dimensional Bayesian inverse problems

Maximum-a-posteriori estimation with Bayesian confidence regions

Bayesian Inference for Brain Activity from Functional Magnetic Resonance Imaging Collected at Two Spatial Resolutions

Nonlinear Deconvolution by Sampling Biophysically Plausible Hemodynamic Models