Abstract:Approximation of solutions to partial differential equations (PDE) is an important problem in computational science and engineering. Using neural networks as an ansatz for the solution has proven a challenge in terms of training time and approximation accuracy. In this contribution, we discuss how sampling the hidden weights and biases of the ansatz network from data-agnostic and data-dependent probability distributions allows us to progress on both challenges. In most examples, the random sampling schemes outperform iterative, gradient-based optimization of physics-informed neural networks regarding training time and accuracy by several orders of magnitude. For time-dependent PDE, we construct neural basis functions only in the spatial domain and then solve the associated ordinary differential equation with classical methods from scientific computing over a long time horizon. This alleviates one of the greatest challenges for neural PDE solvers because it does not require us to parameterize the solution in time. For second-order elliptic PDE in Barron spaces, we prove the existence of sampled networks with $L^2$ convergence to the solution. We demonstrate our approach on several time-dependent and static PDEs. We also illustrate how sampled networks can effectively solve inverse problems in this setting. Benefits compared to common numerical schemes include spectral convergence and mesh-free construction of basis functions.

What problem does this paper attempt to address?

### Problems the paper attempts to solve This paper aims to solve the problem of approximate solutions to partial differential equations (PDEs). Specifically, the paper explores how to improve the neural - network - based PDE - solving methods by sampling the hidden weights and biases in neural networks. It mainly focuses on the following aspects: 1. **Training time and accuracy**: When using a neural network as an approximate function (ansatz) for PDE solutions, long training time and low accuracy are challenges. The paper proposes a method of sampling hidden weights and biases from data - independent and data - dependent probability distributions to reduce training time and improve accuracy. 2. **Solving time - dependent PDEs**: For time - dependent PDEs, traditional methods usually treat the time dimension as another spatial dimension, resulting in the need to model changes in time and space. The paper proposes a method of separating variables, that is, constructing neural basis functions in the spatial domain and then using classical scientific computing methods to solve the related ordinary differential equations (ODEs), thus avoiding the need for time parameterization. 3. **Convergence of second - order elliptic PDEs**: For second - order elliptic PDEs defined in Barron spaces, the paper proves that there exist sampling networks whose L2 norms converge to the solutions of PDEs. 4. **Solving inverse problems**: Inverse problems refer to the situation where some parameters in PDEs are unknown and only sparse solution measurements are available. The paper proposes two methods to simultaneously solve the unknown parameters of PDEs and the solutions over the entire domain. One method is suitable for low - dimensional parameter spaces, and the other is suitable for infinite - dimensional parameter fields. ### Main contributions - **Sampling methods**: Two sampling methods - extreme learning machines (ELM) and sampling important regions (SWIM) - are proposed to generate the hidden - layer weights and biases of neural networks. - **Time - separation method**: For time - dependent PDEs, a variable - separation method is adopted to decompose the problem into spatial and temporal parts, thereby simplifying the solving process. - **Theoretical guarantees**: It is proved that the sampling networks can converge to the solutions of PDEs under specific conditions. - **Inverse - problem solving**: Effective methods are proposed to handle inverse problems, especially for optimization problems in high - dimensional parameter fields. ### Experimental results The paper verifies the effectiveness of the proposed methods through multiple experiments, including the linear advection equation, the fourth - order Euler - Bernoulli beam equation, the nonlinear viscous Burgers equation, and the Poisson and Helmholtz inverse problems. The experimental results show that the proposed sampling methods are superior to traditional physics - informed neural networks (PINNs) and other numerical methods in both training time and accuracy. ### Summary By introducing new sampling methods and time - separation techniques, this paper significantly improves the efficiency and accuracy of neural - network - based PDE - solving methods. These methods are applicable not only to forward problems but also to inverse problems and have broad application prospects.

Solving partial differential equations with sampled neural networks

Non-equispaced Fourier Neural Solvers for PDEs

A Discussion on Solving Partial Differential Equations using Neural Networks

Learning Partial Differential Equations from Data Using Neural Networks

Solving Partial Differential Equations Using Artificial Neural Networks

A Priori Estimation of the Approximation, Optimization and Generalization Errors of Random Neural Networks for Solving Partial Differential Equations

Solving parametric PDE problems with artificial neural networks

Neural Q-learning for solving PDEs

Solving Partial Differential Equations Using Point-Based Neural Networks.

Semi-Implicit Neural Solver for Time-dependent Partial Differential Equations

Monte Carlo Neural PDE Solver for Learning PDEs via Probabilistic Representation

Quasi-Monte Carlo sampling for machine-learning partial differential equations

Neural networks catching up with finite differences in solving partial differential equations in higher dimensions

Adversarial Sampling for Solving Differential Equations with Neural Networks

Solving High-Dimensional Partial Differential Equations Using Deep Learning.

Neural Q-learning for solving elliptic PDEs

Dynamical Measure Transport and Neural PDE Solvers for Sampling

Adversarial Adaptive Sampling: Unify PINN and Optimal Transport for the Approximation of PDEs

Solving Partial Differential Equations Using Deep Learning and Physical Constraints

Solutions to Elliptic and Parabolic Problems via Finite Difference Based Unsupervised Small Linear Convolutional Neural Networks

Neural-Network-Assisted Finite Difference Discretization for Numerical Solution of Partial Differential Equations