Abstract:Operator learning has become a powerful tool in machine learning for modeling complex physical systems governed by partial differential equations (PDEs). Although Deep Operator Networks (DeepONet) show promise, they require extensive data acquisition. Physics-informed DeepONets (PI-DeepONet) mitigate data scarcity but suffer from inefficient training processes. We introduce Separable Operator Networks (SepONet), a novel framework that significantly enhances the efficiency of physics-informed operator learning. SepONet uses independent trunk networks to learn basis functions separately for different coordinate axes, enabling faster and more memory-efficient training via forward-mode automatic differentiation. We provide a universal approximation theorem for SepONet proving that it generalizes to arbitrary operator learning problems, and then validate its performance through comprehensive benchmarking against PI-DeepONet. Our results demonstrate SepONet's superior performance across various nonlinear and inseparable PDEs, with SepONet's advantages increasing with problem complexity, dimension, and scale. For 1D time-dependent PDEs, SepONet achieves up to $112\times$ faster training and $82\times$ reduction in GPU memory usage compared to PI-DeepONet, while maintaining comparable accuracy. For the 2D time-dependent nonlinear diffusion equation, SepONet efficiently handles the complexity, achieving a 6.44\% mean relative $\ell_{2}$ test error, while PI-DeepONet fails due to memory constraints. This work paves the way for extreme-scale learning of continuous mappings between infinite-dimensional function spaces. Open source code is available at \url{<a class="link-external link-https" href="https://github.com/HewlettPackard/separable-operator-networks" rel="external noopener nofollow">this https URL</a>}.

What problem does this paper attempt to address?

This paper attempts to address the issues of training efficiency and memory usage in Physics-Informed Operator Learning, specifically with existing Deep Operator Networks (DeepONet) and Physics-Informed Deep Operator Networks (PI-DeepONet). Specifically: 1. **Data Requirement Issue**: Traditional DeepONet requires a large number of input-output function pairs for supervised training, which can be very expensive or even impractical in some applications (such as high Reynolds number turbulence simulations). 2. **Training Efficiency Issue**: Although PI-DeepONet reduces the need for large training datasets by introducing physics loss, its training process is still very time-consuming and memory-intensive, especially when dealing with multi-configuration partial differential equations (PDEs). To address these issues, the authors propose the Separated Operator Network (SepONet), a new framework that significantly improves the training efficiency and memory usage of physics-informed operator learning by independently learning the basis functions of different coordinate axes. The main contributions of SepONet include: 1. **Improved Training Efficiency and Memory Usage**: SepONet achieves faster training speed and lower memory consumption through Forward-Mode Automatic Differentiation. 2. **Theoretical Foundation**: Provides a universal approximation theorem for SepONet, proving that it can approximate any nonlinear continuous operator to any desired accuracy. 3. **Experimental Validation**: Demonstrates the superior performance of SepONet through benchmark tests with PI-DeepONet on a series of 1D and 2D time-dependent PDEs, particularly excelling as the problem complexity, dimension, and scale increase. In summary, this paper aims to address the bottlenecks of training efficiency and memory usage in existing physics-informed operator learning methods by proposing the SepONet framework, thereby enabling the learning of large-scale continuous mappings.

Separable Operator Networks

Separable DeepONet: Breaking the Curse of Dimensionality in Physics-Informed Machine Learning

Learning the solution operator of parametric partial differential equations with physics-informed DeepONets

Physics-Informed Deep Neural Operator Networks

What do physics-informed DeepONets learn? Understanding and improving training for scientific computing applications

DeepONet for Solving PDEs: Generalization Analysis in Sobolev Training

FB-HyDON: Parameter-Efficient Physics-Informed Operator Learning of Complex PDEs via Hypernetwork and Finite Basis Domain Decomposition

Learning the solution operator of a nonlinear parabolic equation using physics informed deep operator network

Learning nonlinear operators via DeepONet based on the universal approximation theorem of operators

Operator Learning Enhanced Physics-informed Neural Networks for Solving Partial Differential Equations Characterized by Sharp Solutions

Multifidelity Deep Operator Networks For Data-Driven and Physics-Informed Problems

DOSnet as a Non-Black-Box PDE Solver: When Deep Learning Meets Operator Splitting

DeepONet as a Multi-Operator Extrapolation Model: Distributed Pretraining with Physics-Informed Fine-Tuning

Approximation and Generalization of DeepONets for Learning Operators Arising from a Class of Singularly Perturbed Problems

Sequential Deep Operator Networks (S-DeepONet) for predicting full-field solutions under time-dependent loads

Physics-Informed Tailored Finite Point Operator Network for Parametric Interface Problems

Learning Generic Solutions for Multiphase Transport in Porous Media via the Flux Functions Operator

Improved architectures and training algorithms for deep operator networks

R-adaptive DeepONet: Learning Solution Operators for PDEs with Discontinuous Solutions Using an R-adaptive Strategy

Predictions of Transient Vector Solution Fields with Sequential Deep Operator Network

Physics-Informed Neural Operator for Learning Partial Differential Equations