Abstract:Machine learning has witnessed substantial growth, leading to the development of advanced artificial intelligence models crafted to address a wide range of real-world challenges spanning various domains, such as computer vision, natural language processing, and scientific computing. Nevertheless, the creation of custom models for each new task remains a resource-intensive undertaking, demanding considerable computational time and memory resources. In this study, we introduce the concept of the Neural Combinatorial Wavelet Neural Operator (NCWNO) as a foundational model for scientific computing. This model is specifically designed to excel in learning from a diverse spectrum of physics and continuously adapt to the solution operators associated with parametric partial differential equations (PDEs). The NCWNO leverages a gated structure that employs local wavelet experts to acquire shared features across multiple physical systems, complemented by a memory-based ensembling approach among these local wavelet experts. This combination enables rapid adaptation to new challenges. The proposed foundational model offers two key advantages: (i) it can simultaneously learn solution operators for multiple parametric PDEs, and (ii) it can swiftly generalize to new parametric PDEs with minimal fine-tuning. The proposed NCWNO is the first foundational operator learning algorithm distinguished by its (i) robustness against catastrophic forgetting, (ii) the maintenance of positive transfer for new parametric PDEs, and (iii) the facilitation of knowledge transfer across dissimilar tasks. Through an extensive set of benchmark examples, we demonstrate that the NCWNO can outperform task-specific baseline operator learning frameworks with minimal hyperparameter tuning at the prediction stage. We also show that with minimal fine-tuning, the NCWNO performs accurate combinatorial learning of new parametric PDEs.

Wavelet Neural Operator for solving parametric partial differential equations in computational mechanics problems

Wavelet neural operator: a neural operator for parametric partial differential equations

Waveformer for modelling dynamical systems

U-WNO: U-Net Enhanced Wavelet Neural Operator for Solving Parametric Partial Differential Equations

A foundational neural operator that continuously learns without forgetting

Neuroscience inspired scientific machine learning (Part-2): Variable spiking wavelet neural operator

Learning Semilinear Neural Operators : A Unified Recursive Framework For Prediction And Data Assimilation

Wavelet Diffusion Neural Operator

Diffeomorphism Neural Operator for various domains and parameters of partial differential equations

Coupled Multiwavelet Neural Operator Learning for Coupled Partial Differential Equations

Koopman neural operator as a mesh-free solver of non-linear partial differential equations

Local neural operator for solving transient partial differential equations on varied domains

Neural Operator: Learning Maps Between Function Spaces

Learning nonlinear operators in latent spaces for real-time predictions of complex dynamics in physical systems

Neural Operator induced Gaussian Process framework for probabilistic solution of parametric partial differential equations

Long-time Integration of Nonlinear Wave Equations with Neural Operators

Neural Dynamical Operator: Continuous Spatial-Temporal Model with Gradient-Based and Derivative-Free Optimization Methods

Dynamic Gaussian Graph Operator: Learning parametric partial differential equations in arbitrary discrete mechanics problems

Pseudo-Differential Neural Operator: Generalized Fourier Neural Operator for Learning Solution Operators of Partial Differential Equations