Abstract:Physics-informed neural networks (PINNs) have emerged as powerful tools for solving a wide range of partial differential equations (PDEs). However, despite their user-friendly interface and broad applicability, PINNs encounter challenges in accurately resolving PDEs, especially when dealing with singular cases that may lead to unsatisfactory local minima. To address these challenges and improve solution accuracy, we propose an innovative approach called Annealed Adaptive Importance Sampling (AAIS) for computing the discretized PDE residuals of the cost functions, inspired by the Expectation Maximization algorithm used in finite mixtures to mimic target density. Our objective is to approximate discretized PDE residuals by strategically sampling additional points in regions with elevated residuals, thus enhancing the effectiveness and accuracy of PINNs. Implemented together with a straightforward resampling strategy within PINNs, our AAIS algorithm demonstrates significant improvements in efficiency across a range of tested PDEs, even with limited training datasets. Moreover, our proposed AAIS-PINN method shows promising capabilities in solving high-dimensional singular PDEs. The adaptive sampling framework introduced here can be integrated into various PINN frameworks.

What problem does this paper attempt to address?

The problem that this paper attempts to solve is: How to improve the accuracy and efficiency of physics - informed neural networks (PINNs) when solving high - dimensional partial differential equations (PDEs), especially when dealing with complex PDEs with singularities. Traditional PINNs methods are prone to fall into local minima when facing multimodal loss landscapes or multi - scale PDEs, resulting in a decline in the quality of solutions. To solve these problems, the paper proposes a new adaptive sampling method - Annealed Adaptive Importance Sampling (AAIS) to enhance the training effect of PINNs. ### Specific Problems and Solutions 1. **Problem Background**: - **Importance of PDEs**: PDEs are used to model various physical phenomena. Although traditional numerical solvers are accurate, they are computationally expensive and difficult to handle complex systems. - **Advantages and Challenges of PINNs**: PINNs encode PDEs into the loss function through an automatic differentiation mechanism, having the advantages of being easy to code, having a mesh - free structure, and avoiding the curse of dimensionality. However, PINNs are prone to fall into local minima when dealing with complex PDEs, especially performing poorly in multimodal loss landscapes or multi - scale problems. 2. **Proposed Solutions**: - **AAIS Method**: The paper introduces the AAIS method, which is based on the Expectation - Maximization (EM) algorithm and improves the training of PINNs by adaptive sampling in regions with high residuals. Specifically, the AAIS method gradually adds new components and adjusts the mixing weights to better approximate the target distribution. - **Adaptive Sampling Framework**: The paper also proposes a simple resampling framework, which generates new sampling points during the training process and keeps the size of the training data set fixed, thereby improving the training efficiency. ### Main Contributions 1. **Proposing a New Adaptive Sampling Method**: - **AAIS - PINNs**: It includes two methods based on Gaussian mixture (AAIS - g) and Student's t - distribution (AAIS - t). These methods can more accurately capture the multimodal residuals in high - dimensional PDEs, significantly improving the accuracy of solving complex PDEs. - **Robust Resampling Framework**: A simple and effective resampling framework is proposed, which can maintain the consistent size of the training data set and reduce the risk of local minima through adaptive sampling points. 2. **Experimental Verification**: - **Numerical Experiments**: Through numerical experiments on multiple PDEs, the effectiveness of the AAIS - PINNs method is verified. The experimental results show that the AAIS method performs well in both low - and high - dimensional PDEs, especially having an obvious advantage when dealing with multimodal PDEs. - **Performance Comparison**: Compared with methods such as uniform sampling (Uni) and residual - based adaptive distribution (RAD), the AAIS method is particularly prominent in high - dimensional problems and can obtain better solutions with limited computational resources. ### Conclusion By introducing the AAIS method, the paper significantly improves the accuracy and efficiency of PINNs when solving high - dimensional PDEs, especially when dealing with multimodal loss landscapes and multi - scale problems. The proposed adaptive sampling framework and resampling strategy provide new solutions for PINNs in practical applications and have broad application prospects.

Annealed adaptive importance sampling method in PINNs for solving high dimensional partial differential equations

Enhancing PINNs for solving PDEs via adaptive collocation point movement and adaptive loss weighting

Failure-informed adaptive sampling for PINNs

A comprehensive study of non-adaptive and residual-based adaptive sampling for physics-informed neural networks

Revisiting PINNs: Generative Adversarial Physics-informed Neural Networks and Point-weighting Method

An Adaptive Sampling Algorithm with Dynamic Iterative Probability Adjustment Incorporating Positional Information

A Spatial-Temporally Adaptive PINN Framework for 3D Bi-Ventricular Electrophysiological Simulations and Parameter Inference

GAS: A Gaussian Mixture Distribution-Based Adaptive Sampling Method for PINNs

TCAS-PINN: Physics-informed neural networks with a novel temporal causality-based adaptive sampling method

A Dimension-Augmented Physics-Informed Neural Network (DaPINN) with High Level Accuracy and Efficiency

A-PINN: Auxiliary physics informed neural networks for forward and inverse problems of nonlinear integro-differential equations

Investigating Guiding Information for Adaptive Collocation Point Sampling in PINNs

Adaptive Sampling Points Based Multi-Scale Residual Network for Solving Partial Differential Equations

Efficient training of physics-informed neural networks via importance sampling

RANG: A Residual-based Adaptive Node Generation Method for Physics-Informed Neural Networks

Causality-guided adaptive sampling method for physics-informed neural networks

Physics-informed neural networks with residual/gradient-based adaptive sampling methods for solving PDEs with sharp solutions

Efficient training of physics‐informed neural networks via importance sampling

Physics Informed Neural Networks (PINNs) as intelligent computing technique for solving partial differential equations: Limitation and Future prospects

Randomized Physics-Informed Neural Networks for Bayesian Data Assimilation

Adaptive transfer learning for PINN