Abstract:We propose a novel modular inference approach combining two different generative models — generative adversarial networks (GAN) and normalizing flows — to approximate the posterior distribution of physics-based Bayesian inverse problems framed in high-dimensional ambient spaces. We dub the proposed framework GAN-Flow. The proposed method leverages the intrinsic dimension reduction and superior sample generation capabilities of GANs to define a low-dimensional data-driven prior distribution. Once a trained GAN-prior is available, the inverse problem is solved entirely in the latent space of the GAN using variational Bayesian inference with normalizing flow-based variational distribution, which approximates low-dimensional posterior distribution by transforming realizations from the low-dimensional latent prior (Gaussian) to corresponding realizations of a low-dimensional variational posterior distribution. The trained GAN generator then maps realizations from this approximate posterior distribution in the latent space back to the high-dimensional ambient space. We also propose a two-stage training strategy for GAN-Flow wherein we train the two generative models sequentially. Thereafter, GAN-Flow can estimate the statistics of posterior-predictive quantities of interest at virtually no additional computational cost. The synergy between the two types of generative models allows us to overcome many challenges associated with the application of Bayesian inference to large-scale inverse problems, chief among which are describing an informative prior and sampling from the high-dimensional posterior. GAN-Flow does not involve Markov chain Monte Carlo simulation, making it particularly suitable for solving large-scale inverse problems. We demonstrate the efficacy and flexibility of GAN-Flow on various physics-based inverse problems of varying ambient dimensionality and prior knowledge using different types of GANs and normalizing flows. Notably, one of the applications we consider involves a 65,536-dimensional inverse problem of phase retrieval wherein an object is reconstructed from sparse noisy measurements of the magnitude of its Fourier transform.

NF-ULA: Normalizing Flow-Based Unadjusted Langevin Algorithm for Imaging Inverse Problems

NF-ULA: Langevin Monte Carlo with Normalizing Flow Prior for Imaging Inverse Problems

Normalizing flow regularization for photoacoustic tomography

Bayesian Inference and Deep Learning for Inverse Problems

Preconditioned training of normalizing flows for variational inference in inverse problems

Functional normalizing flow for statistical inverse problems of partial differential equations

Faster Uncertainty Quantification for Inverse Problems with Conditional Normalizing Flows

Parallel Backpropagation for Inverse of a Convolution with Application to Normalizing Flows

Normalizing flow sampling with Langevin dynamics in the latent space

Normalizing field flows: Solving forward and inverse stochastic differential equations using physics-informed flow models

Flow Annealed Kalman Inversion for Gradient-Free Inference in Bayesian Inverse Problems

Normalising Flows for Bayesian Gravity Inversion

Reduced-order Model-Based Variational Inference with Normalizing Flows for Bayesian Elliptic Inverse Problems

Accelerated Bayesian Imaging by Relaxed Proximal-Point Langevin Sampling

NLTG Priors in Medical Image: Nonlocal TV-Gaussian (NLTG) prior for Bayesian inverse problems with applications to Limited CT Reconstruction

Nonlocal TV-Gaussian Prior for Bayesian Inverse Problems with Applications to Limited CT Reconstruction

Learning to solve Bayesian inverse problems: An amortized variational inference approach using Gaussian and Flow guides

Solving High-dimensional Inverse Problems Using Amortized Likelihood-free Inference with Noisy and Incomplete Data

A dimension-reduced variational approach for solving physics-based inverse problems using generative adversarial network priors and normalizing flows