Abstract:The classical development of neural networks has primarily focused on learning mappings between finite dimensional Euclidean spaces or finite sets. We propose a generalization of neural networks to learn operators, termed neural operators, that map between infinite dimensional function spaces. We formulate the neural operator as a composition of linear integral operators and nonlinear activation functions. We prove a universal approximation theorem for our proposed neural operator, showing that it can approximate any given nonlinear continuous operator. The proposed neural operators are also discretization-invariant, i.e., they share the same model parameters among different discretization of the underlying function spaces. Furthermore, we introduce four classes of efficient parameterization, viz., graph neural operators, multi-pole graph neural operators, low-rank neural operators, and Fourier neural operators. An important application for neural operators is learning surrogate maps for the solution operators of partial differential equations (PDEs). We consider standard PDEs such as the Burgers, Darcy subsurface flow, and the Navier-Stokes equations, and show that the proposed neural operators have superior performance compared to existing machine learning based methodologies, while being several orders of magnitude faster than conventional PDE solvers.

Theoretical Research on Neural Networks in Application to Dynamic System Modeling

A neurofuzzy network structure for nonlinear system modeling

Application study of a kind of novel neural networks models

Approximation Capability of a Novel Neural Network Model for Dynamic Systems

A New Neural Network Model And Its Application

Approximations of Continuous Functionals by Neural Networks with Application to Dynamic Systems

Neural Network Approximations of Compositional Functions With Applications to Dynamical Systems

Universal Approximation to Nonlinear Operators by Neural Networks with Arbitrary Activation Functions and Its Application to Dynamical Systems

A critical look at deep neural network for dynamic system modeling

Universal Approximation of Multiple Nonlinear Operators by Neural Networks

Neural Operator: Learning Maps Between Function Spaces

Standard neural network model and its application

Capability of Neural Networks in Computing the Outputs of Dynamic Systems with Inputs Defined on the Whole Space

A Process Neural Network for Continuous Process Approximation and Its Application

Basis Operator Network: A Neural Network-Based Model for Learning Nonlinear Operators Via Neural Basis

Neural network modeling and dynamic behavior prediction of nonlinear dynamic systems

A Mathematical Analysis of Neural Operator Behaviors

Some Theoretical Issues on Continuous Process Neural Networks

The Chaos Theory and Its Application in Neural Network Modeling

Dynamical systems' based neural networks