Abstract:We propose a scalable preconditioned primal-dual hybrid gradient algorithm for solving partial differential equations (PDEs). We multiply the PDE with a dual test function to obtain an inf-sup problem whose loss functional involves lower-order differential operators. The Primal-Dual Hybrid Gradient (PDHG) algorithm is then leveraged for this saddle point problem. By introducing suitable precondition operators to the proximal steps in the PDHG algorithm, we obtain an alternative natural gradient ascent-descent optimization scheme for updating the neural network parameters. We apply the Krylov subspace method (MINRES) to evaluate the natural gradients efficiently. Such treatment readily handles the inversion of precondition matrices via matrix-vector multiplication. A posterior convergence analysis is established for the time-continuous version of the proposed method. The algorithm is tested on various types of PDEs with dimensions ranging from $1$ to $50$, including linear and nonlinear elliptic equations, reaction-diffusion equations, and Monge-Ampère equations stemming from the $L^2$ optimal transport problems. We compare the performance of the proposed method with several commonly used deep learning algorithms such as physics-informed neural networks (PINNs), the DeepRitz method, weak adversarial networks (WANs), etc, for solving PDEs using the Adam and L-BFGS optimizers. The numerical results suggest that the proposed method performs efficiently and robustly and converges more stably.

What problem does this paper attempt to address?

### What problems does this paper attempt to solve? This paper aims to solve several key challenges encountered when using traditional deep - learning methods to solve partial differential equations (PDEs). Specifically, the author proposes a new optimization algorithm - the Natural Primal - Dual Hybrid Gradient (NPDHG) method to address the following issues: 1. **Optimization stability**: - Traditional optimizers such as Adam and RMSProp are prone to violent fluctuations when dealing with nonlinear neural networks, resulting in unstable convergence. - The paper improves the stability and convergence speed of the optimization process by introducing the pre - conditioned natural gradient method. 2. **Computational efficiency for high - dimensional problems**: - When the dimension of PDEs is high, the computational cost of traditional methods increases dramatically, making it difficult to solve efficiently. - The NPDHG method effectively deals with the solution of high - dimensional PDEs by combining the pre - conditioning matrix and Krylov subspace methods (such as MINRES). 3. **Loss function design and hyperparameter adjustment**: - In the process of deep - learning to solve PDEs, the design of the loss function and the selection of hyperparameters are crucial, but existing methods are deficient in this regard. - This method provides theoretical convergence guarantees and verifies its effectiveness and robustness on different types of PDEs (including linear and nonlinear elliptic equations, reaction - diffusion equations, and Monge - Ampère equations) through experiments. 4. **Handling of saddle - point problems**: - The weak form of PDEs can usually be transformed into saddle - point problems, and existing deep - learning methods are not efficient enough in dealing with such problems. - The NPDHG method effectively solves saddle - point problems by introducing appropriate pre - conditioning operators and natural gradients, and is applicable to multiple types of PDEs. 5. **Solution of nonlinear PDEs**: - The solution of nonlinear PDEs is a difficult problem in scientific computing, especially for the fully nonlinear Monge - Ampère equation. - The paper shows that the NPDHG method is not only applicable to linear PDEs, but can also be extended to handle nonlinear and even fully nonlinear PDEs, such as the Monge - Ampère equation. ### Summary This paper proposes a new NPDHG method, aiming to improve the stability and efficiency of deep - learning in solving PDEs, especially in high - dimensional and nonlinear cases. By introducing the pre - conditioned natural gradient and optimization strategies for saddle - point problems, this method performs well in multiple test cases and outperforms existing commonly - used methods such as PINNs, DeepRitz method, and WAN.

A Natural Primal-Dual Hybrid Gradient Method for Adversarial Neural Network Training on Solving Partial Differential Equations

A Non-Gradient Method for Solving Elliptic Partial Differential Equations with Deep Neural Networks.

Primal-dual hybrid gradient algorithms for computing time-implicit Hamilton-Jacobi equations

End-to-End Mesh Optimization of a Hybrid Deep Learning Black-Box PDE Solver

Dual Cone Gradient Descent for Training Physics-Informed Neural Networks

TENG: Time-Evolving Natural Gradient for Solving PDEs With Deep Neural Nets Toward Machine Precision

Neural Q-learning for solving elliptic PDEs

Neural Q-learning for solving PDEs

Accelerated primal-dual methods with enlarged step sizes and operator learning for nonsmooth optimal control problems

A first-order computational algorithm for reaction-diffusion type equations via primal-dual hybrid gradient method

Deep Petrov-Galerkin Method for Solving Partial Differential Equations

Solving High-Dimensional Partial Differential Equations Using Deep Learning.

Quantifying Training Difficulty and Accelerating Convergence in Neural Network-Based PDE Solvers

Solving Partial Differential Equations Using Deep Learning and Physical Constraints

RBF-Assisted Hybrid Neural Network for Solving Partial Differential Equations

Physics-informed neural network based on a new adaptive gradient descent algorithm for solving partial differential equations of flow problems

Global Convergence of Deep Galerkin and PINNs Methods for Solving Partial Differential Equations

Accelerated nonlinear primal-dual hybrid gradient methods with applications to supervised machine learning

Numerical solution for high-dimensional partial differential equations based on deep learning with residual learning and data-driven learning

A hybrid physics-informed neural network based multiscale solver as a partial differential equation constrained optimization problem