Abstract:Physics-informed neural networks (PINNs) have shown to be effective tools for solving both forward and inverse problems of partial differential equations (PDEs). PINNs embed the PDEs into the loss of the neural network using automatic differentiation, and this PDE loss is evaluated at a set of scattered spatio-temporal points (called residual points). The location and distribution of these residual points are highly important to the performance of PINNs. However, in the existing studies on PINNs, only a few simple residual point sampling methods have mainly been used. Here, we present a comprehensive study of two categories of sampling for PINNs: non-adaptive uniform sampling and adaptive nonuniform sampling. We consider six uniform sampling methods, including (1) equispaced uniform grid, (2) uniformly random sampling, (3) Latin hypercube sampling, (4) Halton sequence, (5) Hammersley sequence, and (6) Sobol sequence. We also consider a resampling strategy for uniform sampling. To improve the sampling efficiency and the accuracy of PINNs, we propose two new residual-based adaptive sampling methods: residual-based adaptive distribution (RAD) and residual-based adaptive refinement with distribution (RAR-D), which dynamically improve the distribution of residual points based on the PDE residuals during training. Hence, we have considered a total of 10 different sampling methods, including six non-adaptive uniform sampling, uniform sampling with resampling, two proposed adaptive sampling, and an existing adaptive sampling. We extensively tested the performance of these sampling methods for four forward problems and two inverse problems in many setups. Our numerical results presented in this study are summarized from more than 6000 simulations of PINNs. We show that the proposed adaptive sampling methods of RAD and RAR-D significantly improve the accuracy of PINNs with fewer residual points for both forward and inverse problems. The results obtained in this study can also be used as a practical guideline in choosing sampling methods.

Failure-Informed Adaptive Sampling for PINNs, Part III: Applications to Inverse Problems

Failure-informed adaptive sampling for PINNs

A comprehensive study of non-adaptive and residual-based adaptive sampling for physics-informed neural networks

A Spatial-Temporally Adaptive PINN Framework for 3D Bi-Ventricular Electrophysiological Simulations and Parameter Inference

An Adaptive Sampling Algorithm with Dynamic Iterative Probability Adjustment Incorporating Positional Information

Mitigating Propagation Failures in Physics-informed Neural Networks using Retain-Resample-Release (R3) Sampling

Estimates on the generalization error of Physics Informed Neural Networks (PINNs) for approximating a class of inverse problems for PDEs

TCAS-PINN: Physics-informed neural networks with a novel temporal causality-based adaptive sampling method

Annealed adaptive importance sampling method in PINNs for solving high dimensional partial differential equations

Adaptive Self-supervision Algorithms for Physics-informed Neural Networks

Inverse Physics-Informed Neural Networks for transport models in porous materials

A Gaussian Mixture Distribution-Based Adaptive Sampling Method for Physics-Informed Neural Networks

Enhancing PINNs for solving PDEs via adaptive collocation point movement and adaptive loss weighting

Randomized Physics-Informed Neural Networks for Bayesian Data Assimilation

Causality-guided adaptive sampling method for physics-informed neural networks

Data-Guided Physics-Informed Neural Networks for Solving Inverse Problems in Partial Differential Equations

Recipes for when physics fails: recovering robust learning of physics informed neural networks

Physics-informed graph neural Galerkin networks: A unified framework for solving PDE-governed forward and inverse problems

Spatiotemporal parallel physics-informed neural networks: A framework to solve inverse problems in fluid mechanics

An E-PINN assisted practical uncertainty quantification for inverse problems

Pre-Training Physics-Informed Neural Network with Mixed Sampling and Its Application in High-Dimensional Systems