Abstract:This work proposes a solution for the problem of training physics-informed networks under partial integro-differential equations. These equations require an infinite or a large number of neural evaluations to construct a single residual for training. As a result, accurate evaluation may be impractical, and we show that naive approximations at replacing these integrals with unbiased estimates lead to biased loss functions and solutions. To overcome this bias, we investigate three types of potential solutions: the deterministic sampling approaches, the double-sampling trick, and the delayed target method. We consider three classes of PDEs for benchmarking; one defining Poisson problems with singular charges and weak solutions of up to 10 dimensions, another involving weak solutions on electro-magnetic fields and a Maxwell equation, and a third one defining a Smoluchowski coagulation problem. Our numerical results confirm the existence of the aforementioned bias in practice and also show that our proposed delayed target approach can lead to accurate solutions with comparable quality to ones estimated with a large sample size integral. Our implementation is open-source and available at <a class="link-external link-https" href="https://github.com/ehsansaleh/btspinn" rel="external noopener nofollow">this https URL</a>.

What problem does this paper attempt to address?

### Problems the paper attempts to solve This paper aims to solve the training problems of Physics - Informed Neural Networks (PINNs) when dealing with Partial Integro - Differential Equations (PIDEs). Specifically, PIDEs require a large number of neural network evaluations in each training step to construct a single residual, which may be impractical in real - world applications. Moreover, using simple unbiased estimates to approximate these integrals will lead to biased loss functions and solutions. #### Main challenges 1. **Computational complexity**: Accurately evaluating the residuals of PIDE may require an infinite or a large number of neural network evaluations. 2. **Bias problem**: Using simple unbiased estimates to approximate integrals will lead to biased loss functions and solutions, thus affecting the model performance. ### Proposed solutions To solve the above problems, the authors propose three potential solutions: 1. **Deterministic Sampling Approaches**: Eliminate additional variance terms by deterministically selecting sample points. However, this method will re - define the optimization loss and may lead to bias. 2. **Double - Sampling Trick**: Eliminate bias by independently sampling two sample points. But this method requires access to multiple independent and identically distributed samples, which is not feasible in some cases. 3. **Delayed Target Method**: Based on the idea of temporal difference learning, introduce a delayed target model to reduce the bias and variance in the parameter gradients. This method can also produce results comparable to large - sample - size integral estimates in the case of a single sample. ### Experimental verification The authors carried out experimental verification through the following three types of partial differential equations (PDEs): 1. **Poisson Problems**: Involving singular charges and weak solutions, up to 10 dimensions. 2. **Maxwell Equation**: Involving weak solutions of electromagnetic fields. 3. **Smoluchowski Coagulation Problem**: Simulating the evolution dynamics of particles. The experimental results show that the Delayed Target Method can produce results comparable to large - sample - size integral estimates in the case of a single sample and can effectively overcome the bias problem. ### Summary This paper proposes a new method to solve the training problems encountered by PINNs when dealing with PIDEs, especially how to obtain accurate solutions under limited computational resources. By introducing the Delayed Target Method, the authors successfully reduce the bias and improve the model performance.

Learning from Integral Losses in Physics Informed Neural Networks

Critical Investigation of Failure Modes in Physics-informed Neural Networks

Recipes for when physics fails: recovering robust learning of physics informed neural networks

An Analysis of Physics-Informed Neural Networks

Learning a Neural Solver for Parametric PDE to Enhance Physics-Informed Methods

Physics-informed neural networks: A deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations

Characterizing and Mitigating the Difficulty in Training Physics-informed Artificial Neural Networks under Pointwise Constraints

Inverse-Dirichlet Weighting Enables Reliable Training of Physics Informed Neural Networks

Solving differential equations using physics informed deep learning: a hand-on tutorial with benchmark tests

Data-Guided Physics-Informed Neural Networks for Solving Inverse Problems in Partial Differential Equations

Physics Informed Deep Learning (Part I): Data-driven Solutions of Nonlinear Partial Differential Equations

Physics-informed learning of governing equations from scarce data

Optimally weighted loss functions for solving PDEs with Neural Networks

Is L 2 Physics-Informed Loss Always Suitable for Training Physics-Informed Neural Network?

Integration of physics-informed operator learning and finite element method for parametric learning of partial differential equations

Enhanced physics‐informed neural networks for hyperelasticity

Physics-Informed Neural Networks with Adaptive Localized Artificial Viscosity

Physics Constrained Learning for Data-driven Inverse Modeling from Sparse Observations

Learning Physics-Informed Neural Networks Without Stacked Back-propagation.

About optimal loss function for training physics-informed neural networks under respecting causality

A trial solution for imposing boundary conditions of partial differential equations in physics-informed neural networks