Abstract:We study the combination of the alternating direction method of multipliers (ADMM) with physics-informed neural networks (PINNs) for a general class of nonsmooth partial differential equation (PDE)-constrained optimization problems, where additional regularization can be employed for constraints on the control or design variables. The resulting ADMM-PINNs algorithmic framework substantially enlarges the applicable range of PINNs to nonsmooth cases of PDE-constrained optimization problems. The application of the ADMM makes it possible to untie the PDE constraints and the nonsmooth regularization terms for iterations. Accordingly, at each iteration, one of the resulting subproblems is a smooth PDE-constrained optimization which can be efficiently solved by PINNs, and the other is a simple nonsmooth optimization problem which usually has a closed-form solution or can be efficiently solved by various standard optimization algorithms or pre-trained neural networks. The ADMM-PINNs algorithmic framework does not require to solve PDEs repeatedly, and it is mesh-free, easy to implement, and scalable to different PDE settings. We validate the efficiency of the ADMM-PINNs algorithmic framework by different prototype applications, including inverse potential problems, source identification in elliptic equations, control constrained optimal control of the Burgers equation, and sparse optimal control of parabolic equations.

What problem does this paper attempt to address?

The problem that this paper attempts to solve is the **non - smooth partial differential equation (PDE) - constrained optimization problem**. Such problems are widely used in many scientific and engineering fields, such as optimal control, inverse problems, shape design, etc. Specifically, the paper focuses on how to combine the alternating direction method of multipliers (ADMM) with the physics - informed neural network (PINNs) to propose a new algorithm framework (ADMM - PINNs) in order to efficiently solve the PDE - constrained optimization problem with non - smooth regularization terms. ### Problem Background PDE - constrained optimization problems usually involve finding a control variable \(u\) such that the objective functional \(J(y, u)+R(u)\) is minimized while satisfying the PDE constraint \(e(y, u) = 0\). Here, \(y\) is the state variable, \(J(y, u)\) is the data fidelity term and possible smooth regularization terms, and \(R(u)\) is the non - smooth regularization term used to capture the prior information of the control variable \(u\), such as sparsity, discontinuity and upper and lower bound constraints. ### Challenges Traditional numerical methods face multiple challenges when dealing with such problems: 1. **Non - smoothness**: The non - smooth regularization term \(R(u)\) prevents the direct application of classical optimization algorithms such as gradient descent. 2. **High - dimensionality**: Solving high - dimensional PDEs requires a large amount of computing resources. 3. **Ill - conditioning**: Grid - based numerical discretization methods will lead to large - scale and ill - conditioned algebraic systems. ### Solutions To overcome these challenges, the paper proposes the ADMM - PINNs algorithm framework. The main features of this framework include: 1. **ADMM**: Through ADMM, the smooth PDE constraint and the non - smooth regularization term are processed separately, so that each sub - problem can be efficiently solved separately. 2. **PINNs**: Use PINNs to approximate the state variable and the control variable in the PDE - constrained optimization problem, and solve the PDE by minimizing the loss function. ### Specific Steps 1. **ADMM Decomposition**: Decompose the original problem into two sub - problems, one involving the smooth PDE constraint and the other involving the non - smooth regularization term. 2. **PINNs Solution**: Use PINNs to solve the smooth PDE - constrained sub - problem. 3. **Non - smooth Sub - problem Solution**: Non - smooth sub - problems usually have closed - form solutions or can be efficiently solved by standard optimization algorithms. ### Application Examples The paper verifies the effectiveness of the ADMM - PINNs algorithm framework in different application scenarios, including: - **Inverse potential problem** - **Control - constrained optimal control of Burgers' equation** - **Source identification of elliptic equations** - **Sparse optimal control of parabolic equations** ### Summary The ADMM - PINNs algorithm framework provides an efficient and flexible method for solving PDE - constrained optimization problems with non - smooth regularization terms. This framework not only avoids the high computational cost of traditional methods, but also can maintain good performance under multiple PDE types and non - smooth regularization terms.

The ADMM-PINNs Algorithmic Framework for Nonsmooth PDE-Constrained Optimization: A Deep Learning Approach

Enhancing PINNs for solving PDEs via adaptive collocation point movement and adaptive loss weighting

An Efficient Framework for Solving Forward and Inverse Problems of Nonlinear Partial Differential Equations Via Enhanced Physics-Informed Neural Network Based on Adaptive Learning

Physical informed neural networks with soft and hard boundary constraints for solving advection-diffusion equations using Fourier expansions

Bi-level Physics-Informed Neural Networks for PDE Constrained Optimization using Broyden's Hypergradients

A hybrid physics-informed neural network based multiscale solver as a partial differential equation constrained optimization problem

MultiAdam: Parameter-wise Scale-invariant Optimizer for Multiscale Training of Physics-informed Neural Networks

SK-PINN: Accelerated physics-informed deep learning by smoothing kernel gradients

A-PINN: Auxiliary physics informed neural networks for forward and inverse problems of nonlinear integro-differential equations

An Improved Physics-Informed Neural Network Based on a New Adaptive Gradient Descent Algorithm for Solving Partial Differential Equations of Nonlinear Systems

Enhancing neurodynamic approach with physics-informed neural networks for solving non-smooth convex optimization problems

Physics-informed neural networks based on adaptive weighted loss functions for Hamilton-Jacobi equations

Hybrid Finite Difference with the Physics-informed Neural Network for solving PDE in complex geometries

The Hard-Constraint PINNs for Interface Optimal Control Problems

NAS-PINN: Neural architecture search-guided physics-informed neural network for solving PDEs

Navigating PINNs via maximum residual-based continuous distribution

Monte Carlo PINNs: deep learning approach for forward and inverse problems involving high dimensional fractional partial differential equations

Deep NURBS -- Admissible Physics-informed Neural Networks

Physics-informed Neural Networks for Advection–diffusion–langmuir Adsorption Processes

NSGA-PINN: A Multi-Objective Optimization Method for Physics-Informed Neural Network Training