Abstract:For multi-scale problems, the conventional physics-informed neural networks (PINNs) face some challenges in obtaining available predictions. In this paper, based on PINNs, we propose a practical deep learning framework for multi-scale problems by reconstructing the loss function and associating it with specialized neural network architectures. New PINN methods derived from the improved PINN framework differ from the conventional PINN method mainly in two aspects. First, the new methods use a novel loss function by modifying the standard loss function through a (grouping) regularization strategy. The regularization strategy implements a different power operation on each loss term so that all loss terms composing the loss function are of approximately the same order of magnitude, which makes all loss terms be optimized synchronously during the optimization process. Second, for the multi-frequency or high-frequency problems, in addition to using the modified loss function, new methods upgrade the neural network architecture from the common fully-connected neural network to specialized network architectures such as the Fourier feature architecture given in Ref. [1] and the integrated architecture developed by us. The combination of the above two techniques leads to a significant improvement in the computational accuracy of multi-scale problems. Several challenging numerical examples demonstrate the effectiveness of the proposed methods. The proposed methods not only significantly outperform the conventional PINN method in terms of computational efficiency and computational accuracy, but also compare favorably with the state-of-the-art methods in the recent literature. The improved PINN framework facilitates better application of PINNs to multi-scale problems. The data and code accompanying this paper are available at https://github.com/wangyong1301108/MMPINN .

Improved physics-informed neural network in mitigating gradient related failures

A Spatial-Temporally Adaptive PINN Framework for 3D Bi-Ventricular Electrophysiological Simulations and Parameter Inference

A simple remedy for failure modes in physics informed neural networks

Recipes for when physics fails: recovering robust learning of physics informed neural networks

Neuroevolution of Physics-Informed Neural Nets: Benchmark Problems and Comparative Results

Residual-based attention and connection to information bottleneck theory in PINNs

Adaptive Self-supervision Algorithms for Physics-informed Neural Networks

Enhancing PINNs for solving PDEs via adaptive collocation point movement and adaptive loss weighting

An Improved Physics-Informed Neural Network Based on a New Adaptive Gradient Descent Algorithm for Solving Partial Differential Equations of Nonlinear Systems

Auto-PINN: Understanding and Optimizing Physics-Informed Neural Architecture

Gradient-optimized physics-informed neural networks (GOPINNs): a deep learning method for solving the complex modified KdV equation

SK-PINN: Accelerated physics-informed deep learning by smoothing kernel gradients

A Dimension-Augmented Physics-Informed Neural Network (DaPINN) with High Level Accuracy and Efficiency

Adaptive transfer learning for PINN

Generalized conditional symmetry enhanced physics-informed neural network and application to the forward and inverse problems of nonlinear diffusion equations

An Expert's Guide to Training Physics-informed Neural Networks

Improving PINNs By Algebraic Inclusion of Boundary and Initial Conditions

ANaGRAM: A Natural Gradient Relative to Adapted Model for efficient PINNs learning

ST-PINN: A Self-Training Physics-Informed Neural Network for Partial Differential Equations

Learning Physics-Informed Neural Networks without Stacked Back-propagation

A practical PINN framework for multi-scale problems with multi-magnitude loss terms