Abstract:In recent years, advances in computing hardware and computational methods have prompted a wealth of activities for solving inverse problems in physics. These problems are often described by systems of partial differential equations (PDEs). The advent of machine learning has reinvigorated the interest in solving inverse problems using neural networks (NNs). In these efforts, the solution of the PDEs is expressed as NNs trained through the minimization of a loss function involving the PDE. Here, we show how to accelerate this approach by five orders of magnitude by deploying, instead of NNs, conventional PDE approximations. The framework of optimizing a discrete loss (ODIL) minimizes a cost function for discrete approximations of the PDEs using gradient-based and Newton's methods. The framework relies on grid-based discretizations of PDEs and inherits their accuracy, convergence, and conservation properties. The implementation of the method is facilitated by adopting machine-learning tools for automatic differentiation. We also propose a multigrid technique to accelerate the convergence of gradient-based optimizers. We present applications to PDE-constrained optimization, optical flow, system identification, and data assimilation. We compare ODIL with the popular method of physics-informed neural networks and show that it outperforms it by several orders of magnitude in computational speed while having better accuracy and convergence rates. We evaluate ODIL on inverse problems involving linear and nonlinear PDEs including the Navier-Stokes equations for flow reconstruction problems. ODIL bridges numerical methods and machine learning and presents a powerful tool for solving challenging, inverse problems across scientific domains.

Solving ill-posed inverse problems using iterative deep neural networks

Bayesian Inference and Deep Learning for Inverse Problems

Joint learning of variational representations and solvers for inverse problems with partially-observed data

Deep neural networks can stably solve high-dimensional, noisy, non-linear inverse problems

An Over Complete Deep Learning Method for Inverse Problems

Solving Inverse Problems in Steady State Navier-Stokes Equations using Deep Neural Networks

Deep Convolutional Neural Network for Inverse Problems in Imaging

Neumann Networks for Inverse Problems in Imaging

Deep Learning Methods for Solving Linear Inverse Problems: Research Directions and Paradigms

Regularization of Inverse Problems by Neural Networks

DRIP: Deep Regularizers for Inverse Problems

Neural Inverse Operators for Solving PDE Inverse Problems

Big in Japan: Regularizing networks for solving inverse problems

A Fast Data-Driven Iteratively Regularized Method with Convex Penalty for Solving Ill-Posed Problems

Deep Learning and Inverse Problems

Model-Aware Regularization For Learning Approaches To Inverse Problems

To be or not to be stable, that is the question: understanding neural networks for inverse problems

Spectral Function Space Learning and Numerical Linear Algebra Networks for Solving Linear Inverse Problems

Solving inverse problems in physics by optimizing a discrete loss: Fast and accurate learning without neural networks

Deep Regularized Compound Gaussian Network for Solving Linear Inverse Problems