PINNACLE: PINN Adaptive ColLocation and Experimental points selection

Gregory Kang Ruey Lau,Apivich Hemachandra,See-Kiong Ng,Bryan Kian Hsiang Low

2024-04-11

Abstract:Physics-Informed Neural Networks (PINNs), which incorporate PDEs as soft constraints, train with a composite loss function that contains multiple training point types: different types of collocation points chosen during training to enforce each PDE and initial/boundary conditions, and experimental points which are usually costly to obtain via experiments or simulations. Training PINNs using this loss function is challenging as it typically requires selecting large numbers of points of different types, each with different training dynamics. Unlike past works that focused on the selection of either collocation or experimental points, this work introduces PINN Adaptive ColLocation and Experimental points selection (PINNACLE), the first algorithm that jointly optimizes the selection of all training point types, while automatically adjusting the proportion of collocation point types as training progresses. PINNACLE uses information on the interaction among training point types, which had not been considered before, based on an analysis of PINN training dynamics via the Neural Tangent Kernel (NTK). We theoretically show that the criterion used by PINNACLE is related to the PINN generalization error, and empirically demonstrate that PINNACLE is able to outperform existing point selection methods for forward, inverse, and transfer learning problems.

Machine Learning,Artificial Intelligence,Computational Physics,Data Analysis, Statistics and Probability

What problem does this paper attempt to address?

The paper addresses the challenge of selecting different types of training points in the training process of Physics Informed Neural Networks (PINNs), including points for enforcing Partial Differential Equations (PDEs) and initial/boundary conditions, as well as expensive experimental points obtained through experiments or simulations. Existing works deal with the selection of these points separately, while this paper proposes the first algorithm, named PINNACLE, that simultaneously optimizes the selection of all training point types by automatically adjusting their ratio during the training process. PINNACLE utilizes the analysis of Neural Tangent Kernel (NTK) to analyze the training dynamics, defines a new convergence measure, and theoretically proves that maximizing this measure can reduce the generalization error of PINNs. Experiments show that PINNACLE outperforms existing point selection methods in forward, inverse, and transfer learning problems.

PINNACLE: PINN Adaptive ColLocation and Experimental points selection

Enhancing PINNs for solving PDEs via adaptive collocation point movement and adaptive loss weighting

Investigating Guiding Information for Adaptive Collocation Point Sampling in PINNs

BC-PINN: an adaptive physics informed neural network based on biased multiobjective coevolutionary algorithm

Adaptive Self-supervision Algorithms for Physics-informed Neural Networks

A Spatial-Temporally Adaptive PINN Framework for 3D Bi-Ventricular Electrophysiological Simulations and Parameter Inference

CP-PINNs: Data-Driven Changepoints Detection in PDEs Using Online Optimized Physics-Informed Neural Networks

Loss-attentional physics-informed neural networks

PACMANN: Point Adaptive Collocation Method for Artificial Neural Networks

Revisiting PINNs: Generative Adversarial Physics-informed Neural Networks and Point-weighting Method

RoPINN: Region Optimized Physics-Informed Neural Networks

GPT-PINN: Generative Pre-Trained Physics-Informed Neural Networks toward non-intrusive Meta-learning of parametric PDEs

Architectural Strategies for the optimization of Physics-Informed Neural Networks

Adaptive transfer learning for PINN

iPINNs: incremental learning for Physics-informed neural networks

PINNacle: A Comprehensive Benchmark of Physics-Informed Neural Networks for Solving PDEs

A practical PINN framework for multi-scale problems with multi-magnitude loss terms

Failure-informed adaptive sampling for PINNs

A Dimension-Augmented Physics-Informed Neural Network (DaPINN) with High Level Accuracy and Efficiency

Physics-informed PointNet: On how many irregular geometries can it solve an inverse problem simultaneously? Application to linear elasticity