Abstract:Physics-informed neural networks (PINNs) are revolutionizing science and engineering practice by bringing together the power of deep learning to bear on scientific computation. In forward modeling problems, PINNs are meshless partial differential equation (PDE) solvers that can handle irregular, high-dimensional physical domains. Naturally, the neural architecture hyperparameters have a large impact on the efficiency and accuracy of the PINN solver. However, this remains an open and challenging problem because of the large search space and the difficulty of identifying a proper search objective for PDEs. Here, we propose Auto-PINN, the first systematic, automated hyperparameter optimization approach for PINNs, which employs Neural Architecture Search (NAS) techniques to PINN design. Auto-PINN avoids manually or exhaustively searching the hyperparameter space associated with PINNs. A comprehensive set of pre-experiments using standard PDE benchmarks allows us to probe the structure-performance relationship in PINNs. We find that the different hyperparameters can be decoupled, and that the training loss function of PINNs is a good search objective. Comparison experiments with baseline methods demonstrate that Auto-PINN produces neural architectures with superior stability and accuracy over alternative baselines.

What problem does this paper attempt to address?

### Problems Addressed by the Paper This paper aims to address the issue of neural architecture hyperparameter optimization in Physics-Informed Neural Networks (PINNs). Specifically, the paper proposes Auto-PINN, a systematic automatic hyperparameter optimization method designed to find the optimal PINN architecture and hyperparameters in a large-scale search space. #### Main Issues: 1. **Challenges of Hyperparameter Optimization**: Hyperparameters in PINNs significantly impact the efficiency and accuracy of solutions, but current methods of manually or exhaustively searching these hyperparameters are very time-consuming and difficult to implement. 2. **Suitable Search Objective Function**: For Partial Differential Equations (PDEs), finding a suitable search objective function is not easy because many PDEs do not have exact solutions, making it impossible to directly use error values as the objective. #### Solutions: 1. **Pre-experiment Analysis**: Through a series of benchmark tests, the relationship between different hyperparameters and performance is studied, and it is found that some design choices (such as activation functions) can be decoupled from the search space. 2. **Automatic Optimization Framework**: The Auto-PINN framework is proposed, adopting a step-by-step decoupling strategy to reduce the search space and using training loss values as the search objective. 3. **Experimental Validation**: Through multiple PDE benchmark tests, Auto-PINN is validated to be superior to other baseline methods in terms of accuracy and stability. #### Main Contributions: 1. **Pre-experiment Findings**: It is found that certain hyperparameters (such as activation functions) can be easily decoupled from the search space, and training loss values are identified as an effective search objective. 2. **Auto-PINN Framework**: A new automatic hyperparameter optimization method is proposed, significantly reducing the search complexity. 3. **Experimental Results**: It is demonstrated that Auto-PINN outperforms other baseline methods in multiple PDE benchmark tests, achieving better accuracy and stability with fewer search iterations.

Auto-PINN: Understanding and Optimizing Physics-Informed Neural Architecture

AutoPINN: When AutoML Meets Physics-Informed Neural Networks

NAS-PINN: Neural architecture search-guided physics-informed neural network for solving PDEs

Auto-PICNN: Automated machine learning for physics-informed convolutional neural networks

Architectural Strategies for the optimization of Physics-Informed Neural Networks

A Spatial-Temporally Adaptive PINN Framework for 3D Bi-Ventricular Electrophysiological Simulations and Parameter Inference

An Expert's Guide to Training Physics-informed Neural Networks

RoPINN: Region Optimized Physics-Informed Neural Networks

Machine Learning Through Physics–Informed Neural Networks: Progress and Challenges

Improved physics-informed neural network in mitigating gradient related failures

Loss-attentional physics-informed neural networks

NSGA-PINN: A Multi-Objective Optimization Method for Physics-Informed Neural Network Training

Physics-informed Neural Networks (PINN) for computational solid mechanics: Numerical frameworks and applications

Fine-Tuning DeepONets to Enhance Physics-informed Neural Networks for solving Partial Differential Equations

On the Generalization of PINNs outside the training domain and the Hyperparameters influencing it

Revisiting PINNs: Generative Adversarial Physics-informed Neural Networks and Point-weighting Method

A Dimension-Augmented Physics-Informed Neural Network (DaPINN) with High Level Accuracy and Efficiency

ST-PINN: A Self-Training Physics-Informed Neural Network for Partial Differential Equations

Augmented Physics-Informed Neural Networks (APINNs): A gating network-based soft domain decomposition methodology

Adaptive Self-supervision Algorithms for Physics-informed Neural Networks