Abstract:Physics-Informed Neural Networks (PINNs) are a class of deep neural networks that are trained, using automatic differentiation, to compute the response of systems governed by partial differential equations (PDEs). The training of PINNs is simulation-free, and does not require any training dataset to be obtained from numerical PDE solvers. Instead, it only requires the physical problem description, including the governing laws of physics, domain geometry, initial/boundary conditions, and the material properties. This training usually involves solving a non-convex optimization problem using variants of the stochastic gradient descent method, with the gradient of the loss function approximated on a batch of collocation points, selected randomly in each iteration according to a uniform distribution. Despite the success of PINNs in accurately solving a wide variety of PDEs, the method still requires improvements in terms of computational efficiency. To this end, in this paper, we study the performance of an importance sampling approach for efficient training of PINNs. Using numerical examples together with theoretical evidences, we show that in each training iteration, sampling the collocation points according to a distribution proportional to the loss function will improve the convergence behavior of the PINNs training. Additionally, we show that providing a piecewise constant approximation to the loss function for faster importance sampling can further improve the training efficiency. This importance sampling approach is straightforward and easy to implement in the existing PINN codes, and also does not introduce any new hyperparameter to calibrate. The numerical examples include elasticity, diffusion and plane stress problems, through which we numerically verify the accuracy and efficiency of the importance sampling approach compared to the predominant uniform sampling approach.

Meshfree Variational Physics Informed Neural Networks (MF-VPINN): an adaptive training strategy

Robust Variational Physics-Informed Neural Networks

Variational Physics-Informed Neural Networks For Solving Partial Differential Equations

Collocation-based Robust Variational Physics-Informed Neural Networks (CRVPINN)

Physics-Informed Neural Networks with Adaptive Localized Artificial Viscosity

FastVPINNs: Tensor-Driven Acceleration of VPINNs for Complex Geometries

Cv-Pinn: Efficient Learning of Variational Physics-Informed Neural Network with Domain Decomposition

VI-PINNs: Variance-involved Physics-informed Neural Networks for Fast and Accurate Prediction of Partial Differential Equations

VW-PINNs: A volume weighting method for PDE residuals in physics-informed neural networks

Accelerated Training of Physics-Informed Neural Networks (PINNs) using Meshless Discretizations

Effects of variational formulations on physics‐informed neural network performance in solid mechanics

Finite Volume Physical Informed Neural Network (FV-PINN) with Reduced Derivative Order for Incompressible Flows

Extremization to Fine Tune Physics Informed Neural Networks for Solving Boundary Value Problems

Inverse-Dirichlet Weighting Enables Reliable Training of Physics Informed Neural Networks

Adaptive-Sampling Physics-Informed Neural Network for Viscoacoustic Wavefield Simulation

Recipes for when physics fails: recovering robust learning of physics informed neural networks

Inverse Physics-Informed Neural Networks for transport models in porous materials

VS-PINN: A fast and efficient training of physics-informed neural networks using variable-scaling methods for solving PDEs with stiff behavior

Efficient training of physics-informed neural networks via importance sampling

Improving hp-Variational Physics-Informed Neural Networks for Steady-State Convection-Dominated Problems

An efficient hp-Variational PINNs framework for incompressible Navier-Stokes equations