Abstract:Neural operator architectures approximate operators between infinite-dimensional Banach spaces of functions. They are gaining increased attention in computational science and engineering, due to their potential both to accelerate traditional numerical methods and to enable data-driven discovery. As the field is in its infancy basic questions about minimal requirements for universal approximation remain open. It is clear that any general approximation of operators between spaces of functions must be both nonlocal and nonlinear. In this paper we describe how these two attributes may be combined in a simple way to deduce universal approximation. In so doing we unify the analysis of a wide range of neural operator architectures and open up consideration of new ones. A popular variant of neural operators is the Fourier neural operator (FNO). Previous analysis proving universal operator approximation theorems for FNOs resorts to use of an unbounded number of Fourier modes, relying on intuition from traditional analysis of spectral methods. The present work challenges this point of view: (i) the work reduces FNO to its core essence, resulting in a minimal architecture termed the ``averaging neural operator'' (ANO); and (ii) analysis of the ANO shows that even this minimal ANO architecture benefits from universal approximation. This result is obtained based on only a spatial average as its only nonlocal ingredient (corresponding to retaining only a \emph{single} Fourier mode in the special case of the FNO). The analysis paves the way for a more systematic exploration of nonlocality, both through the development of new operator learning architectures and the analysis of existing and new architectures. Numerical results are presented which give insight into complexity issues related to the roles of channel width (embedding dimension) and number of Fourier modes.

Learning -matrix phases with neural operators

Learning S-Matrix Phases with Neural Operators

Scattering with Neural Operators

Discretization Error of Fourier Neural Operators

3D elastic wave propagation with a Factorized Fourier Neural Operator (F-FNO)

Fourier Neural Operators for Learning Dynamics in Quantum Spin Systems

Fourier Neural Operator Surrogate Model to Predict 3D Seismic Waves Propagation

Fourier Neural Differential Equations for learning Quantum Field Theories

Understanding the Expressivity and Trainability of Fourier Neural Operator: A Mean-Field Perspective

Nonlocality and Nonlinearity Implies Universality in Operator Learning

Physics-enhanced Neural Network for Phase Retrieval from Two Diffraction Patterns

Multi-Resolution Active Learning of Fourier Neural Operators

Full phase space resonant anomaly detection

Bounding the Rademacher complexity of Fourier neural operators

Learning New Physics from Data -- a Symmetrized Approach

Domain Agnostic Fourier Neural Operators

Model-Parallel Fourier Neural Operators as Learned Surrogates for Large-Scale Parametric PDEs

Divergence-free neural operators for stress field modeling in polycrystalline materials

Phase-amplitude formalism for ultra-narrow shape resonances

Lorentz meets Fano spectral line shapes: A universal phase and its laser control

A spectral method for a Fokker-Planck equation in neuroscience with applications in neural networks with learning rules