Abstract:Although physics-informed neural networks (PINNs) have shown great potential in dealing with nonlinear partial differential equations (PDEs), it is common that PINNs will suffer from the problem of insufficient precision or obtaining incorrect outcomes. Unlike most of the existing solutions trying to enhance the ability of PINN by optimizing the training process, this paper improved the neural network architecture to improve the performance of PINN. We propose a densely multiply PINN (DM-PINN) architecture, which multiplies the output of a hidden layer with the outputs of all the behind hidden layers. Without introducing more trainable parameters, this effective mechanism can significantly improve the accuracy of PINNs. The proposed architecture is evaluated on four benchmark examples (Allan-Cahn equation, Helmholtz equation, Burgers equation and 1D convection equation). Comparisons between the proposed architecture and different PINN structures demonstrate the superior performance of the DM-PINN in both accuracy and efficiency.

What problem does this paper attempt to address?

The problem that this paper attempts to solve is that Physics - Informed Neural Networks (PINNs) often face the problems of insufficient accuracy or incorrect results when dealing with nonlinear partial differential equations (PDEs). Specifically, PINNs may encounter the following challenges when solving PDEs: 1. **Unbalanced Residual Distribution**: When calculating the loss function, some points with large residuals may account for only a small part of the entire training data set, resulting in the importance of these points being ignored and thus affecting the accuracy of the final solution. 2. **Space - Time Causality Problem**: Different from traditional numerical calculation methods, PINNs lack the guarantee of causality during the training process. That is, all time and space points are input into the neural network simultaneously, and it cannot be ensured that the problems of early time points are solved first and then the problems of later time points are solved. 3. **Simple Structure**: Traditional PINNs adopt simple fully - connected neural networks (FC - NN), and this structure may have limitations when expressing complex physical phenomena. To solve these problems, the author proposes a new neural network architecture - Densely Multiplied PINN (DM - PINN). By introducing element - level multiplication operations between hidden - layer outputs, DM - PINN can significantly improve the accuracy and performance of PINNs without increasing the number of trainable parameters. In addition, in order to deal with the vanishing gradient problem and further improve the model performance, the author also introduces Batch Normalization layers and ResNet structures (residual blocks), and proposes Skip Densely Multiplied PINN (SDM - PINN). The paper verifies the effectiveness of the proposed method through four benchmark test cases (Allan - Cahn equation, Helmholtz equation, Burgers equation, and 1D convection equation) and compares it with existing PINN structures, demonstrating the superiority of DM - PINN in terms of accuracy and efficiency.

Densely Multiplied Physics Informed Neural Networks

Element-wise Multiplication Based Deeper Physics-Informed Neural Networks

Binary structured physics-informed neural networks for solving equations with rapidly changing solutions

Machine Learning Through Physics–Informed Neural Networks: Progress and Challenges

Physics Informed Neural Networks (PINNs) as intelligent computing technique for solving partial differential equations: Limitation and Future prospects

MRF-PINN: A Multi-Receptive-Field convolutional physics-informed neural network for solving partial differential equations

Learning Physics-Informed Neural Networks without Stacked Back-propagation

PINNsFormer: A Transformer-Based Framework For Physics-Informed Neural Networks

Distributed physics informed neural network for data-efficient solution to partial differential equations

A Dimension-Augmented Physics-Informed Neural Network (DaPINN) with High Level Accuracy and Efficiency

iPINNs: incremental learning for Physics-informed neural networks

A Physics Informed Neural Network for Time-Dependent Nonlinear and Higher Order Partial Differential Equations

Scientific Machine Learning Through Physics–Informed Neural Networks: Where we are and What's Next

Physics-informed neural networks combined with polynomial interpolation to solve nonlinear partial differential equations

Recipes for when physics fails: recovering robust learning of physics informed neural networks

Physics-informed attention-based neural network for solving non-linear partial differential equations

ST-PINN: A Self-Training Physics-Informed Neural Network for Partial Differential Equations

Accelerating Physics-Informed Neural Network Training with Prior Dictionaries

SVD-PINNs: Transfer Learning of Physics-Informed Neural Networks via Singular Value Decomposition

Parallel Physics-Informed Neural Networks with Bidirectional Balance

SetPINNs: Set-based Physics-informed Neural Networks