Abstract:Neural operators, as a powerful approximation to the non-linear operators between infinite-dimensional function spaces, have proved to be promising in accelerating the solution of partial differential equations (PDE). However, it requires a large amount of simulated data, which can be costly to collect. This can be avoided by learning physics from the physics-constrained loss, which we refer to it as mean squared residual (MSR) loss constructed by the discretized PDE. We investigate the physical information in the MSR loss, which we called long-range entanglements, and identify the challenge that the neural network requires the capacity to model the long-range entanglements in the spatial domain of the PDE, whose patterns vary in different PDEs. To tackle the challenge, we propose LordNet, a tunable and efficient neural network for modeling various entanglements. Inspired by the traditional solvers, LordNet models the long-range entanglements with a series of matrix multiplications, which can be seen as the low-rank approximation to the general fully-connected layers and extracts the dominant pattern with reduced computational cost. The experiments on solving Poisson's equation and (2D and 3D) Navier-Stokes equation demonstrate that the long-range entanglements from the MSR loss can be well modeled by the LordNet, yielding better accuracy and generalization ability than other neural networks. The results show that the Lordnet can be $50\times$ faster than traditional PDE solvers. In addition, LordNet outperforms other modern neural network architectures in accuracy and efficiency with the smallest parameter size.

Energy stable neural network for gradient flow equations

Stable Neural Flows

Energetic Variational Neural Network Discretizations of Gradient Flows

Efficient and stable SAV-based methods for gradient flows arising from deep learning

Deep Learning via Neural Energy Descent

Global-in-time energy stability: a powerful analysis tool for the gradient flow problem without maximum principle or Lipschitz assumption

Solving Euler equations with gradient-weighted multi-input high-dimensional feature neural network

Cell-average based neural network method for hyperbolic and parabolic partial differential equations

Physics-Informed Neural Networks for Solving Coupled Stokes-Darcy Equation

Predicting unsteady incompressible fluid dynamics with finite volume informed neural network

Energy-dissipative spectral renormalization exponential integrator method for gradient flow problems

Evolutional deep neural network

Entropy stable conservative flux form neural networks

An Adaptive Energy-Based Sequential Method for Training PINNs to Solve Gradient Flow Equations

Finite Difference Neural Networks: Fast Prediction of Partial Differential Equations

Energy-Dissipative Evolutionary Deep Operator Neural Networks

LordNet: An Efficient Neural Network for Learning to Solve Parametric Partial Differential Equations without Simulated Data

Deep Latent Regularity Network for Modeling Stochastic Partial Differential Equations

A Stabilized Physics Informed Neural Networks Method for Wave Equations

A High Resolution Physics-informed Neural Networks for High-Dimensional Convection-Diffusion-reaction Equations.