Abstract:Physics-Informed Neural Networks (PINNs) have emerged as a powerful tool for integrating physics-based constraints and data to address forward and inverse problems in machine learning. Despite their potential, the implementation of PINNs are hampered by several challenges, including issues related to convergence, stability, and the design of neural networks and loss functions. In this paper, we introduce a novel training scheme that addresses these challenges by framing the training process as a constrained optimization problem. Utilizing a quadratic program (QP)-based gradient descent law, our approach simplifies the design of loss functions and guarantees convergences to optimal neural network parameters. This methodology enables dynamic balancing, over the course of training, between data-based loss and a partial differential equation (PDE) residual loss, ensuring an acceptable level of accuracy while prioritizing the minimization of PDE-based loss. We demonstrate the formulation of the constrained PINNs approach with noisy data, in the context of solving Laplace's equation in a capacitor with complex geometry. This work not only advances the capabilities of PINNs but also provides a framework for their training.

What problem does this paper attempt to address?

This paper attempts to solve several key problems encountered in the training process of Physics - Informed Neural Networks (PINNs), which include: 1. **Convergence and stability**: Existing PINN methods may be difficult to converge or be unstable during the optimization process. 2. **Loss function design**: The loss function of PINNs usually consists of data loss and residual loss based on physical equations (such as partial differential equations, PDEs). When there are conflicts between these loss terms, it will lead to optimization difficulties. 3. **The influence of noisy data**: In practical applications, measurement data often contains noise, which may cause the model to over - fit the noise at the expense of compliance with physical laws. To solve these problems, the author proposes a new training scheme, which regards the training process as a constrained optimization problem and utilizes the Quadratic - Program - based Gradient Descent method (QPGD). Specifically, this method ensures minimizing data errors while satisfying physical constraints by dynamically balancing data loss and PDE residual loss. In addition, this method can also handle noisy data and has demonstrated its effectiveness in solving the capacitor problem of Laplace's equation. ### Main contributions 1. **Simplifying loss function design**: By introducing the QP framework, the design of the loss function is simplified, making the trade - off between different loss terms more natural. 2. **Ensuring convergence and stability**: The proposed QPGD method can ensure that the parameters converge to the optimal value and remain stable throughout the training process. 3. **Handling noisy data**: This method can reasonably balance data fitting and physical consistency in the presence of noise and avoid over - fitting the noise. 4. **Verification in practical applications**: Through a case study of a complex capacitor geometry, the effectiveness and superiority of this method in solving forward and inverse problems are verified. ### Summary of mathematical formulas - Formal representation of the constrained optimization problem: \[ \min_{\theta} f(\theta) \quad \text{subject to} \quad g(\theta) \leq 0 \] where \( S=\{\theta \in \mathbb{R}^n:g(\theta) \leq 0\}\), \( f:\mathbb{R}^n \to \mathbb{R}\), \( g:\mathbb{R}^n \to \mathbb{R}\). - Gradient descent update rule: \[ \dot{\theta}(t)=-\nabla f(\theta)=u(\theta) \] - QP - based gradient descent control law: \[ \bar{u}=\arg \min_{v \in \mathbb{R}^n}\|v\|^2 \quad \text{subject to} \quad \nabla g(\theta)^T(u(\theta)+v)+c g(\theta) \leq 0 \] - Discretized parameter update rule: \[ \theta(t + 1)=\theta(t)-\gamma\left(\nabla f(\theta(t))+\alpha(\theta(t))\nabla g(\theta(t))\right) \] where: \[ \alpha(\theta(t))=\max \left\{-\frac{\nabla f(\theta(t))^T\nabla g(\theta(t))+c g(\theta(t))}{\max\{\|\nabla g(\theta(t))\|^2,\epsilon_\alpha\}},0\right\} \] Through these methods, the paper provides a more robust and effective...

QP Based Constrained Optimization for Reliable PINN Training