Abstract:The classical development of neural networks has primarily focused on learning mappings between finite dimensional Euclidean spaces or finite sets. We propose a generalization of neural networks to learn operators, termed neural operators, that map between infinite dimensional function spaces. We formulate the neural operator as a composition of linear integral operators and nonlinear activation functions. We prove a universal approximation theorem for our proposed neural operator, showing that it can approximate any given nonlinear continuous operator. The proposed neural operators are also discretization-invariant, i.e., they share the same model parameters among different discretization of the underlying function spaces. Furthermore, we introduce four classes of efficient parameterization, viz., graph neural operators, multi-pole graph neural operators, low-rank neural operators, and Fourier neural operators. An important application for neural operators is learning surrogate maps for the solution operators of partial differential equations (PDEs). We consider standard PDEs such as the Burgers, Darcy subsurface flow, and the Navier-Stokes equations, and show that the proposed neural operators have superior performance compared to existing machine learning based methodologies, while being several orders of magnitude faster than conventional PDE solvers.

What problem does this paper attempt to address?

The main problem that this paper attempts to solve is the limitations encountered by traditional neural networks when learning mappings between infinite - dimensional function spaces. Specifically, traditional neural networks mainly focus on learning mappings between finite - dimensional Euclidean spaces or finite sets. These methods have the following deficiencies when dealing with complex problems such as partial differential equations (PDEs): 1. **Discretization Dependence**: Traditional neural network architectures are highly dependent on the discretization method of training data. This means that if the discretization grid of the data is changed, the network architecture and parameters may need to be redesigned, which limits the generalization ability of the model. 2. **Inability to Handle Infinite - Dimensional Spaces**: Traditional neural networks can usually only handle finite - dimensional inputs and outputs, while the inputs and outputs in many scientific and engineering problems are infinite - dimensional function spaces. Therefore, traditional methods perform poorly when dealing with these problems. To solve these problems, the paper proposes a new deep - learning framework - **Neural Operators**, which can directly learn mappings between infinite - dimensional function spaces. The key features of neural operators include: - **Discretization Invariance**: Neural operators share the same model parameters on different discretization grids, which means that they can maintain consistent performance on data with different resolutions without retraining. - **Universal Approximation**: The paper proves that the proposed neural operators can approximate any given nonlinear continuous operator, thus having strong expressive and generalization abilities. - **Efficient Parameterization Methods**: The paper proposes four efficient parameterization methods, including Graph Neural Operators, Multi - pole Graph Neural Operators, Low - rank Neural Operators, and Fourier Neural Operators. Through these features, neural operators perform excellently in the task of learning solution operators of partial differential equations. They are not only superior to existing machine - learning methods in accuracy but also far exceed traditional PDE solvers in inference speed. For example, in the solution of the two - dimensional Navier - Stokes equation, the inference time of the Fourier Neural Operator (FNO) is three orders of magnitude faster than that of the pseudo - spectral method, while maintaining high accuracy and robustness in downstream applications.

Neural Operator: Learning Maps Between Function Spaces

Neural Operator: Graph Kernel Network for Partial Differential Equations

Operator Learning: Algorithms and Analysis

Basis Operator Network: A Neural Network-Based Model for Learning Nonlinear Operators Via Neural Basis

A Mathematical Analysis of Neural Operator Behaviors

Neural Operators with Localized Integral and Differential Kernels

Learning nonlinear operators via DeepONet based on the universal approximation theorem of operators

Diffeomorphic Latent Neural Operators for Data-Efficient Learning of Solutions to Partial Differential Equations

Nonlocality and Nonlinearity Implies Universality in Operator Learning

Learning Partial Differential Equations with Deep Parallel Neural Operator

An operator learning perspective on parameter-to-observable maps

Wavelet neural operator: a neural operator for parametric partial differential equations

Learning Neural Operators on Riemannian Manifolds

U-NO: U-shaped Neural Operators

Neural Operators for Accelerating Scientific Simulations and Design

A discretization-invariant extension and analysis of some deep operator networks

Learning integral operators via neural integral equations

Diffeomorphism Neural Operator for various domains and parameters of partial differential equations

Learning nonlinear operators in latent spaces for real-time predictions of complex dynamics in physical systems

Pseudo-Differential Neural Operator: Generalized Fourier Neural Operator for Learning Solution Operators of Partial Differential Equations

Physics-Informed Deep Neural Operator Networks