Abstract:The past decade has seen increasing interest in applying Deep Learning (DL) to Computational Science and Engineering (CSE). Driven by impressive results in applications such as computer vision, Uncertainty Quantification (UQ), genetics, simulations and image processing, DL is increasingly supplanting classical algorithms, and seems poised to revolutionize scientific computing. However, DL is not yet well-understood from the standpoint of numerical analysis. Little is known about the efficiency and reliability of DL from the perspectives of stability, robustness, accuracy, and sample complexity. In particular, approximating solutions to parametric PDEs is an objective of UQ for CSE. Training data for such problems is often scarce and corrupted by errors. Moreover, the target function is a possibly infinite-dimensional smooth function taking values in the PDE solution space, generally an infinite-dimensional Banach space. This paper provides arguments for Deep Neural Network (DNN) approximation of such functions, with both known and unknown parametric dependence, that overcome the curse of dimensionality. We establish practical existence theorems that describe classes of DNNs with dimension-independent architecture size and training procedures based on minimizing the (regularized) $\ell^2$-loss which achieve near-optimal algebraic rates of convergence. These results involve key extensions of compressed sensing for Banach-valued recovery and polynomial emulation with DNNs. When approximating solutions of parametric PDEs, our results account for all sources of error, i.e., sampling, optimization, approximation and physical discretization, and allow for training high-fidelity DNN approximations from coarse-grained sample data. Our theoretical results fall into the category of non-intrusive methods, providing a theoretical alternative to classical methods for high-dimensional approximation.

Optimal deep learning of holomorphic operators between Banach spaces

Near-optimal learning of Banach-valued, high-dimensional functions via deep neural networks

Diffeomorphic Latent Neural Operators for Data-Efficient Learning of Solutions to Partial Differential Equations

Learning in latent spaces improves the predictive accuracy of deep neural operators

Operator Learning: Algorithms and Analysis

Neural Operator: Learning Maps Between Function Spaces

Learning Partial Differential Equations with Deep Parallel Neural Operator

Learning nonlinear operators in latent spaces for real-time predictions of complex dynamics in physical systems

Deep Operator Learning Lessens the Curse of Dimensionality for PDEs

A Mathematical Analysis of Neural Operator Behaviors

An Operator Learning Approach to Nonsmooth Optimal Control of Nonlinear PDEs

Bayesian deep operator learning for homogenized to fine-scale maps for multiscale PDE

Learning smooth functions in high dimensions: from sparse polynomials to deep neural networks

Improved architectures and training algorithms for deep operator networks

Learning nonlinear operators via DeepONet based on the universal approximation theorem of operators

Deep Operator Network Approximation Rates for Lipschitz Operators

Minimax Optimal Kernel Operator Learning via Multilevel Training

Operator Learning of Lipschitz Operators: An Information-Theoretic Perspective

Dynamic Gaussian Graph Operator: Learning parametric partial differential equations in arbitrary discrete mechanics problems

Structure-informed operator learning for parabolic Partial Differential Equations

Finite Operator Learning: Bridging Neural Operators and Numerical Methods for Efficient Parametric Solution and Optimization of PDEs