Abstract:We explore using neural operators, or neural network representations of nonlinear maps between function spaces, to accelerate infinite-dimensional Bayesian inverse problems (BIPs) with models governed by nonlinear parametric partial differential equations (PDEs). Neural operators have gained significant attention in recent years for their ability to approximate the parameter-to-solution maps defined by PDEs using as training data solutions of PDEs at a limited number of parameter samples. The computational cost of BIPs can be drastically reduced if the large number of PDE solves required for posterior characterization are replaced with evaluations of trained neural operators. However, reducing error in the resulting BIP solutions via reducing the approximation error of the neural operators in training can be challenging and unreliable. We provide an a priori error bound result that implies certain BIPs can be ill-conditioned to the approximation error of neural operators, thus leading to inaccessible accuracy requirements in training. To reliably deploy neural operators in BIPs, we consider a strategy for enhancing the performance of neural operators, which is to correct the prediction of a trained neural operator by solving a linear variational problem based on the PDE residual. We show that a trained neural operator with error correction can achieve a quadratic reduction of its approximation error, all while retaining substantial computational speedups of posterior sampling when models are governed by highly nonlinear PDEs. The strategy is applied to two numerical examples of BIPs based on a nonlinear reaction--diffusion problem and deformation of hyperelastic materials. We demonstrate that posterior representations of the two BIPs produced using trained neural operators are greatly and consistently enhanced by error correction.

On the approximation of bi-Lipschitz maps by invertible neural networks

On the Approximation of Bi-Lipschitz Maps by Invertible Neural Networks

On the Approximation and Complexity of Deep Neural Networks to Invariant Functions

Neural Operator: Learning Maps Between Function Spaces

Best Approximation and Inverse Results for Neural Network Operators

Inversion of Integral Models: a Neural Network Approach

Deep Operator Network Approximation Rates for Lipschitz Operators

A discretization-invariant extension and analysis of some deep operator networks

Estimates on the generalization error of Physics Informed Neural Networks (PINNs) for approximating a class of inverse problems for PDEs

Efficient Bayesian inference using physics-informed invertible neural networks for inverse problems

Residual-based error correction for neural operator accelerated infinite-dimensional Bayesian inverse problems

Approximation in Shift-Invariant Spaces with Deep ReLU Neural Networks

Approximations of Continuous Functionals by Neural Networks with Application to Dynamic Systems

Monotone, Bi-Lipschitz, and Polyak-Lojasiewicz Networks

Learning nonlinear operators via DeepONet based on the universal approximation theorem of operators

Sample-Based Continuous Approximate Method for Constructing Interval Neural Network

Approximation and Generalization of DeepONets for Learning Operators Arising from a Class of Singularly Perturbed Problems

Deep Neural Network Approximation of Composition Functions: with Application to PINNs

On the approximation of rough functions with deep neural networks

Can neural operators always be continuously discretized?

Quantitative Approximation for Neural Operators in Nonlinear Parabolic Equations