Abstract:Physics-informed neural networks (PINNs) incorporate physical knowledge from the problem domain as a soft constraint on the loss function, but recent work has shown that this can lead to optimization difficulties. Here, we study the impact of the location of the collocation points on the trainability of these models. We find that the vanilla PINN performance can be significantly boosted by adapting the location of the collocation points as training proceeds. Specifically, we propose a novel adaptive collocation scheme which progressively allocates more collocation points (without increasing their number) to areas where the model is making higher errors (based on the gradient of the loss function in the domain). This, coupled with a judicious restarting of the training during any optimization stalls (by simply resampling the collocation points in order to adjust the loss landscape) leads to better estimates for the prediction error. We present results for several problems, including a 2D Poisson and diffusion-advection system with different forcing functions. We find that training vanilla PINNs for these problems can result in up to 70% prediction error in the solution, especially in the regime of low collocation points. In contrast, our adaptive schemes can achieve up to an order of magnitude smaller error, with similar computational complexity as the baseline. Furthermore, we find that the adaptive methods consistently perform on-par or slightly better than vanilla PINN method, even for large collocation point regimes. The code for all the experiments has been open sourced.

PSO-PINN: Physics-Informed Neural Networks Trained with Particle Swarm Optimization

NSGA-PINN: A Multi-Objective Optimization Method for Physics-Informed Neural Network Training

BC-PINN: an adaptive physics informed neural network based on biased multiobjective coevolutionary algorithm

Revisiting PINNs: Generative Adversarial Physics-informed Neural Networks and Point-weighting Method

Auto-PINN: Understanding and Optimizing Physics-Informed Neural Architecture

CP-PINNs: Data-Driven Changepoints Detection in PDEs Using Online Optimized Physics-Informed Neural Networks

ST-PINN: A Self-Training Physics-Informed Neural Network for Partial Differential Equations

Recipes for when physics fails: recovering robust learning of physics informed neural networks

Enhancing PINNs for solving PDEs via adaptive collocation point movement and adaptive loss weighting

Physics Informed Neural Networks (PINNs) as intelligent computing technique for solving partial differential equations: Limitation and Future prospects

Adaptive Self-supervision Algorithms for Physics-informed Neural Networks

Scientific Machine Learning Through Physics–Informed Neural Networks: Where we are and What's Next

NAS-PINN: Neural architecture search-guided physics-informed neural network for solving PDEs

Pre-training strategy for solving evolution equations based on physics-informed neural networks

Physics-Informed Neural Networks (PINNs) for Parameterized PDEs: A Metalearning Approach

RoPINN: Region Optimized Physics-Informed Neural Networks

iPINNs: incremental learning for Physics-informed neural networks

B-PINNs: Bayesian physics-informed neural networks for forward and inverse PDE problems with noisy data

GPT-PINN: Generative Pre-Trained Physics-Informed Neural Networks toward non-intrusive Meta-learning of parametric PDEs

Learning Physics-Informed Neural Networks without Stacked Back-propagation