Discovering Physics-Informed Neural Networks Model for Solving Partial Differential Equations through Evolutionary Computation

Bo Zhang,Chao Yang

DOI: https://doi.org/10.1016/j.swevo.2024.101589

2024-05-18

Abstract:In recent years, the researches about solving partial differential equations (PDEs) based on artificial neural network have attracted considerable attention. In these researches, the neural network models are usually designed depend on human experience or trial and error. Despite the emergence of several model searching methods, these methods primarily concentrate on optimizing the hyperparameters of fully connected neural network model based on the framework of physics-informed neural networks (PINNs), and the corresponding search spaces are relatively restricted, thereby limiting the exploration of superior models. This article proposes an evolutionary computation method aimed at discovering the PINNs model with higher approximation accuracy and faster convergence rate. In addition to searching the numbers of layers and neurons per hidden layer, this method concurrently explores the optimal shortcut connections between the layers and the novel parametric activation functions expressed by the binary trees. In evolution, the strategy about dynamic population size and training epochs (DPSTE) is adopted, which significantly increases the number of models to be explored and facilitates the discovery of models with fast convergence rate. In experiments, the performance of different models that are searched through Bayesian optimization, random search and evolution is compared in solving Klein-Gordon, Burgers, and Lamé equations. The experimental results affirm that the models discovered by the proposed evolutionary computation method generally exhibit superior approximation accuracy and convergence rate, and these models also show commendable generalization performance with respect to the source term, initial and boundary conditions, equation coefficient and computational domain. The corresponding code is available at

Neural and Evolutionary Computing,Machine Learning

What problem does this paper attempt to address?

The paper aims to address the design problem of neural network models for solving partial differential equations (PDEs). Specifically:

1. **Research Background**: In recent years, the study of using artificial neural networks to solve partial differential equations has received widespread attention. Compared to traditional methods such as finite element and finite difference, neural network-based methods typically do not require mesh division and have stronger capabilities in handling high-dimensional problems. Additionally, experimental or observational data can be seamlessly integrated into the neural network's loss function to improve the accuracy of the solution.

2. **Current Challenges**: Existing Physics-Informed Neural Networks (PINNs) models are usually designed based on researchers' experience or through trial and error. Although there are some model search methods, these methods mainly focus on optimizing hyperparameters within the fully connected neural network framework, resulting in a relatively limited search space and making it difficult to discover better models.

3. **Objectives and Contributions**: This paper proposes an evolutionary computation method to automatically discover PINNs models with higher approximation accuracy and faster convergence speed. The method not only searches for the number of hidden layers and the number of neurons per layer but also explores the optimal shortcut connection structures and new activation functions specific to the problem. By adopting a Dynamic Population Size and Training Epoch strategy (DPSTE), the number of explorable models is significantly increased, aiding in the discovery of models with faster convergence.

In summary, the main objective of this paper is to automatically design better PINNs models through evolutionary algorithms, thereby improving the accuracy and efficiency of solving partial differential equations.