Abstract:We propose a novel modular inference approach combining two different generative models — generative adversarial networks (GAN) and normalizing flows — to approximate the posterior distribution of physics-based Bayesian inverse problems framed in high-dimensional ambient spaces. We dub the proposed framework GAN-Flow. The proposed method leverages the intrinsic dimension reduction and superior sample generation capabilities of GANs to define a low-dimensional data-driven prior distribution. Once a trained GAN-prior is available, the inverse problem is solved entirely in the latent space of the GAN using variational Bayesian inference with normalizing flow-based variational distribution, which approximates low-dimensional posterior distribution by transforming realizations from the low-dimensional latent prior (Gaussian) to corresponding realizations of a low-dimensional variational posterior distribution. The trained GAN generator then maps realizations from this approximate posterior distribution in the latent space back to the high-dimensional ambient space. We also propose a two-stage training strategy for GAN-Flow wherein we train the two generative models sequentially. Thereafter, GAN-Flow can estimate the statistics of posterior-predictive quantities of interest at virtually no additional computational cost. The synergy between the two types of generative models allows us to overcome many challenges associated with the application of Bayesian inference to large-scale inverse problems, chief among which are describing an informative prior and sampling from the high-dimensional posterior. GAN-Flow does not involve Markov chain Monte Carlo simulation, making it particularly suitable for solving large-scale inverse problems. We demonstrate the efficacy and flexibility of GAN-Flow on various physics-based inverse problems of varying ambient dimensionality and prior knowledge using different types of GANs and normalizing flows. Notably, one of the applications we consider involves a 65,536-dimensional inverse problem of phase retrieval wherein an object is reconstructed from sparse noisy measurements of the magnitude of its Fourier transform.

Reduced-order Model-Based Variational Inference with Normalizing Flows for Bayesian Elliptic Inverse Problems

Preconditioned training of normalizing flows for variational inference in inverse problems

Functional normalizing flow for statistical inverse problems of partial differential equations

Bayesian Inference and Deep Learning for Inverse Problems

Learning to solve Bayesian inverse problems: An amortized variational inference approach using Gaussian and Flow guides

Solving High-dimensional Inverse Problems Using Amortized Likelihood-free Inference with Noisy and Incomplete Data

A dimension-reduced variational approach for solving physics-based inverse problems using generative adversarial network priors and normalizing flows

Multiscale model reduction method for Bayesian inverse problems of subsurface flow

Efficient, multimodal, and derivative-free Bayesian inference with Fisher-Rao gradient flows

NF-ULA: Langevin Monte Carlo with Normalizing Flow Prior for Imaging Inverse Problems

Stable Training of Normalizing Flows for High-dimensional Variational Inference

Flow Annealed Kalman Inversion for Gradient-Free Inference in Bayesian Inverse Problems

Bayesian inverse Navier-Stokes problems: joint flow field reconstruction and parameter learning

Normalising Flows for Bayesian Gravity Inversion

Normalizing field flows: Solving forward and inverse stochastic differential equations using physics-informed flow models

Latent Variable Modelling with Hyperbolic Normalizing Flows

A Tale of Two Latent Flows: Learning Latent Space Normalizing Flow with Short-run Langevin Flow for Approximate Inference

Flow Priors for Linear Inverse Problems via Iterative Corrupted Trajectory Matching

Consistency of variational Bayesian inference for non-linear inverse problems of partial differential equations