Characteristic Performance Study on Solving Oscillator ODEs via Soft-constrained Physics-informed Neural Network with Small Data

Kai-liang Lu,Yu-meng Su,Zhuo Bi,Cheng Qiu,Wen-jun Zhang

2024-10-08

Abstract:This paper compared physics-informed neural network (PINN), conventional neural network (NN) and traditional numerical discretization methods on solving differential equations (DEs) through literature investigation and experimental validation. We focused on the soft-constrained PINN approach and formalized its mathematical framework and computational flow for solving Ordinary DEs and Partial DEs (ODEs/PDEs). The working mechanism and its accuracy and efficiency were experimentally verified by solving typical linear and non-linear oscillator ODEs. We demonstrate that the DeepXDE-based implementation of PINN is not only light code and efficient in training, but also flexible across CPU/GPU platforms. PINN greatly reduces the need for labeled data: when the nonlinearity of the ODE is weak, a very small amount of supervised training data plus a few unsupervised collocation points are sufficient to predict the solution; in the minimalist case, only one or two training points (with initial values) are needed for first- or second-order ODEs, respectively. We also find that, with the aid of collocation points and the use of physical information, PINN has the ability to extrapolate data outside the time domain of the training set, and especially is robust to noisy data, thus with enhanced generalization capabilities. Training is accelerated when the gains obtained along with the reduction in the amount of data outweigh the delay caused by the increase in the loss function terms. The soft-constrained PINN can easily impose a physical law (e.g., conservation of energy) constraint by adding a regularization term to the total loss function, thus improving the solution performance to ODEs that obey this physical law. Furthermore, PINN can also be used for stiff ODEs, PDEs, and other types of DEs, and is becoming a favorable catalyst for the era of Digital Twins.

Machine Learning,Computer Vision and Pattern Recognition

What problem does this paper attempt to address?

### What problem does this paper attempt to solve?

This paper primarily explores the application of Physics-Informed Neural Networks (PINN) in solving differential equations, particularly oscillator ordinary differential equations, and demonstrates its advantages over traditional numerical discretization methods and conventional neural networks.

Specifically:
1. **Improvement in scenarios with small data**: In cases of scarce or noisy data, most neural network methods lack robustness and cannot guarantee convergence. PINN improves performance on small datasets by embedding prior knowledge, enhancing generalization ability and interpretability.
2. **Formalization of the soft constraint PINN method**: The framework and computational process of the soft constraint PINN method are studied, verifying its effectiveness, efficiency, and robustness to noise in solving linear and nonlinear oscillator ordinary differential equations.
3. **Comparative analysis**: The advantages and disadvantages of PINN compared to traditional numerical discretization methods (such as the Runge-Kutta method and the finite element method) and conventional neural networks are summarized (see Table 1).
4. **Experimental validation**: By solving typical linear and nonlinear oscillator ordinary differential equations, the performance of PINN with minimal supervised data is demonstrated, especially its ability to predict data outside the training set.
5. **Implementation with DeepXDE**: PINN is implemented based on DeepXDE, proving that this method not only has concise code and high training efficiency but also can flexibly run across CPU/GPU platforms.

In summary, this paper aims to demonstrate, through theoretical analysis and experimental validation, the powerful capability of the soft constraint PINN method in solving differential equations with small data and its advantages in practical applications.