Abstract:Physics-informed neural networks (PINNs) have emerged as an effective technique for solving PDEs in a wide range of domains. It is noticed, however, the performance of PINNs can vary dramatically with different sampling procedures. For instance, a fixed set of (prior chosen) training points may fail to capture the effective solution region (especially for problems with singularities). To overcome this issue, we present in this work an adaptive strategy, termed the failure-informed PINNs (FI-PINNs), which is inspired by the viewpoint of reliability analysis. The key idea is to define an effective failure probability based on the residual, and then, with the aim of placing more samples in the failure region, the FI-PINNs employs a failure-informed enrichment technique to adaptively add new collocation points to the training set, such that the numerical accuracy is dramatically improved. In short, similar as adaptive finite element methods, the proposed FI-PINNs adopts the failure probability as the posterior error indicator to generate new training points. We prove rigorous error bounds of FI-PINNs and illustrate its performance through several problems.

What problem does this paper attempt to address?

The problem that this paper attempts to solve is to improve the performance and accuracy of Physics - Informed Neural Networks (PINNs) when solving Partial Differential Equations (PDEs). Specifically, the paper focuses on how to improve the training effect of PINNs through improved sampling strategies, especially for PDE problems with complex solution structures or singularities, where traditional fixed - sampling methods may not be able to effectively capture the effective solution regions. To solve this problem, the authors propose an adaptive sampling strategy based on the failure probability, called Failure - Informed PINNs (FI - PINNs). ### Main contributions 1. **Define failure probability**: By defining the residual - based failure probability as an error indicator to guide the selection of new training points. 2. **Adaptive sampling technique**: Propose a failure - information - enhanced technique to adaptively add new configuration points to the training set to significantly improve numerical accuracy. 3. **Theoretical analysis**: Prove the strict error bounds of FI - PINNs and verify its performance through experiments on multiple PDE problems. 4. **Efficient sampling method**: Use the truncated Gaussian model to estimate the failure probability and generate new training points, which is more efficient compared to other methods. ### Key concepts - **Failure hyperplane**: Defined by \( g(x)=|r(x; \theta)|-\epsilon_r \), where \( r(x; \theta) \) is the residual and \( \epsilon_r \) is the preset maximum allowable threshold. - **Failure probability**: \( P_F = \int_{\Omega}\omega(x)I_{\Omega_F}(x)dx \), where \( I_{\Omega_F}(x) \) is the indicator function, indicating whether \( x \) belongs to the failure set \( \Omega_F \). - **Importance sampling**: By choosing an appropriate proposal distribution \( h(x) \) to reduce the variance, thus generating samples more effectively. ### Workflow 1. **Initialization**: Set the initial training data set and parameters. 2. **Training**: Train PINNs using the current training data set. 3. **Evaluate failure probability**: Estimate the current failure probability \( \hat{P}_F \). 4. **Judge whether to stop**: If \( \hat{P}_F < \epsilon_p \), stop training; otherwise, generate new training points and update the data set. 5. **Generate new training points**: Generate new training points from the failure region \( \Omega_F \) and add them to the training set. ### Experimental results The paper verifies the effectiveness of FI - PINNs through experiments on multiple PDE problems, including the two - dimensional Poisson equation and the Burgers equation. The experimental results show that FI - PINNs performs excellently in reducing the relative \( L_2 \) error and improving prediction accuracy, especially when dealing with PDE problems with spikes or mutations. In conclusion, this paper significantly improves the performance and accuracy of PINNs in solving complex PDE problems by introducing the failure - information - adaptive sampling strategy.

Failure-informed adaptive sampling for PINNs

A comprehensive study of non-adaptive and residual-based adaptive sampling for physics-informed neural networks

Annealed adaptive importance sampling method in PINNs for solving high dimensional partial differential equations

Enhancing PINNs for solving PDEs via adaptive collocation point movement and adaptive loss weighting

An Adaptive Sampling Algorithm with Dynamic Iterative Probability Adjustment Incorporating Positional Information

Investigating Guiding Information for Adaptive Collocation Point Sampling in PINNs

Adaptive Self-supervision Algorithms for Physics-informed Neural Networks

Revisiting PINNs: Generative Adversarial Physics-informed Neural Networks and Point-weighting Method

TCAS-PINN: Physics-informed neural networks with a novel temporal causality-based adaptive sampling method

Adversarial Training for Physics-Informed Neural Networks

Efficient training of physics-informed neural networks via importance sampling

Self-adaptive loss balanced Physics-informed neural networks

Mitigating Propagation Failures in Physics-informed Neural Networks using Retain-Resample-Release (R3) Sampling

Efficient training of physics‐informed neural networks via importance sampling

Loss-attentional physics-informed neural networks

A Priori Error Estimation of Physics-Informed Neural Networks Solving Allen--Cahn and Cahn--Hilliard Equations

Randomized Physics-Informed Neural Networks for Bayesian Data Assimilation

Recipes for when physics fails: recovering robust learning of physics informed neural networks

RANG: A Residual-based Adaptive Node Generation Method for Physics-Informed Neural Networks

ST-PINN: A Self-Training Physics-Informed Neural Network for Partial Differential Equations

Self-Adaptive Physics-Informed Neural Networks using a Soft Attention Mechanism