Abstract:Operator learning aims to discover properties of an underlying dynamical system or partial differential equation (PDE) from data. Here, we present a step-by-step guide to operator learning. We explain the types of problems and PDEs amenable to operator learning, discuss various neural network architectures, and explain how to employ numerical PDE solvers effectively. We also give advice on how to create and manage training data and conduct optimization. We offer intuition behind the various neural network architectures employed in operator learning by motivating them from the point-of-view of numerical linear algebra.

What problem does this paper attempt to address?

The problem that this paper attempts to solve is: **Discover or approximate an operator of an unknown dynamical system or partial differential equation (PDE) from data**. Specifically, the paper focuses on how to approximate and predict the behaviors of these dynamical systems through machine - learning methods, especially neural network architectures. ### Problem Background In scientific computing and engineering applications, many physical phenomena can be described by partial differential equations (PDEs). However, solving these PDEs is usually very complicated, especially in high - dimensional spaces or in non - linear cases. Although traditional numerical methods are effective, they often require a large amount of computing resources and may be difficult to handle for some complex systems. ### Objectives of Operator Learning Operator Learning aims to directly learn or approximate an unknown operator \(A\) through data - driven methods. This operator is usually related to the solution operator of a differential equation. Specifically, given a set of input - output pairs \((f, u)\), where \(f\in U\) and \(u\in V\) are from function spaces \(U\) and \(V\) respectively, and there is a (possibly non - linear) operator \(A: U\rightarrow V\) such that \(A(f) = u\), the goal is to find an approximate operator \(\hat{A}\) such that for any new input \(f'\in U\), \(\hat{A}(f')\approx A(f')\). ### Key Challenges 1. **Selecting an appropriate neural operator architecture**: Different problems may require different types of neural network architectures to effectively capture the characteristics of the operator. 2. **Optimizing the computational complexity of the problem**: Training neural operators usually involves complex optimization problems and requires efficient algorithms and techniques. 3. **Generalization ability**: Ensure that the model performs well not only on the training data but also can accurately predict unseen data. ### Application Scenarios The applications of operator learning are very extensive, including but not limited to: - **Accelerating numerical PDE solvers**: Accelerate the simulation of complex systems by constructing simplified models. - **Parameter optimization**: Use the approximate solution operator to solve inverse problems and recover unknown parameters. - **Benchmarking new models**: Design specific neural network architectures to preserve important properties in PDEs, such as symmetry and conservation laws. - **Discovering unknown physical laws**: Discover new physical or mathematical models from data and reveal the underlying behaviors of the system. ### Summary The main purpose of this paper is to provide a comprehensive guide to help researchers understand how to use neural network architectures to solve operator - learning problems. By elaborating on different types of problems, PDEs, neural network architectures, as well as data management and optimization strategies, the paper provides a solid foundation for research in this field.

A Mathematical Guide to Operator Learning

Operator Learning: Algorithms and Analysis

A Mathematical Analysis of Neural Operator Behaviors

Operator Learning Meets Numerical Analysis: Improving Neural Networks through Iterative Methods

Neural Operator: Learning Maps Between Function Spaces

An Operator Learning Approach to Nonsmooth Optimal Control of Nonlinear PDEs

An Overview on Machine Learning Methods for Partial Differential Equations: from Physics Informed Neural Networks to Deep Operator Learning

How Analysis Can Teach Us the Optimal Way to Design Neural Operators

Operator Learning with Gaussian Processes

Some Best Practices in Operator Learning

Diffeomorphic Latent Neural Operators for Data-Efficient Learning of Solutions to Partial Differential Equations

Learning Partial Differential Equations with Deep Parallel Neural Operator

Wavelet neural operator: a neural operator for parametric partial differential equations

Solving Partial Differential Equations in Different Domains by Operator Learning method Based on Boundary Integral Equations

Towards Gaussian Process for operator learning: an uncertainty aware resolution independent operator learning algorithm for computational mechanics

Learning Green's functions associated with time-dependent partial differential equations

Variational operator learning: A unified paradigm marrying training neural operators and solving partial differential equations

LeMON: Learning to Learn Multi-Operator Networks

Learning nonlinear operators in latent spaces for real-time predictions of complex dynamics in physical systems

Learning Neural Operators on Riemannian Manifolds

Learning in latent spaces improves the predictive accuracy of deep neural operators