Abstract:Data-driven machine learning approaches are being increasingly used to solve partial differential equations (PDEs). They have shown particularly striking successes when training an operator, which takes as input a PDE in some family, and outputs its solution. However, the architectural design space, especially given structural knowledge of the PDE family of interest, is still poorly understood. We seek to remedy this gap by studying the benefits of weight-tied neural network architectures for steady-state PDEs. To achieve this, we first demonstrate that the solution of most steady-state PDEs can be expressed as a fixed point of a non-linear operator. Motivated by this observation, we propose FNO-DEQ, a deep equilibrium variant of the FNO architecture that directly solves for the solution of a steady-state PDE as the infinite-depth fixed point of an implicit operator layer using a black-box root solver and differentiates analytically through this fixed point resulting in $\mathcal{O}(1)$ training memory. Our experiments indicate that FNO-DEQ-based architectures outperform FNO-based baselines with $4\times$ the number of parameters in predicting the solution to steady-state PDEs such as Darcy Flow and steady-state incompressible Navier-Stokes. Finally, we show FNO-DEQ is more robust when trained with datasets with more noisy observations than the FNO-based baselines, demonstrating the benefits of using appropriate inductive biases in architectural design for different neural network based PDE solvers. Further, we show a universal approximation result that demonstrates that FNO-DEQ can approximate the solution to any steady-state PDE that can be written as a fixed point equation.

Kernel Neural Operator for Efficient Solving PDEs

Neural Operator: Graph Kernel Network for Partial Differential Equations

Pseudo-Differential Neural Operator: Generalized Fourier Neural Operator for Learning Solution Operators of Partial Differential Equations

Neural Operators with Localized Integral and Differential Kernels

Enhancing Solutions for Complex PDEs: Introducing Complementary Convolution and Equivariant Attention in Fourier Neural Operators

Learning Partial Differential Equations with Deep Parallel Neural Operator

Koopman neural operator as a mesh-free solver of non-linear partial differential equations

Neural Operator: Learning Maps Between Function Spaces

Neural Inverse Operators for Solving PDE Inverse Problems

Toward a Better Understanding of Fourier Neural Operators from a Spectral Perspective

Diffeomorphism Neural Operator for various domains and parameters of partial differential equations

Toward Efficient Kernel-Based Solvers for Nonlinear PDEs

Derivative-enhanced Deep Operator Network

U-WNO: U-Net Enhanced Wavelet Neural Operator for Solving Parametric Partial Differential Equations

Gabor-Filtered Fourier Neural Operator for Solving Partial Differential Equations

Convolutional Neural Operators for robust and accurate learning of PDEs

Local neural operator for solving transient partial differential equations on varied domains

Kernel Neural Operators (KNOs) for Scalable, Memory-efficient, Geometrically-flexible Operator Learning

Component Fourier Neural Operator for Singularly Perturbed Differential Equations

Deep Equilibrium Based Neural Operators for Steady-State PDEs