Abstract:In recent years, a significant amount of attention has been paid to solve partial differential equations (PDEs) by deep learning. For example, deep Galerkin method (DGM) uses the PDE residual in the least-squares sense as the loss function and a deep neural network (DNN) to approximate the PDE solution. In this work, we propose a deep mixed residual method (MIM) to solve PDEs with high-order derivatives. Notable examples include Poisson equation, Monge-Ampére equation, biharmonic equation, and Korteweg-de Vries equation. In MIM, we first rewrite a high-order PDE into a first-order system, very much in the same spirit as local discontinuous Galerkin method and mixed finite element method in classical numerical methods for PDEs. We then use the residual of the first-order system in the least-squares sense as the loss function, which is in close connection with least-squares finite element method. For aforementioned classical numerical methods, the choice of trial and test functions is important for stability and accuracy issues in many cases. MIM shares this property when DNNs are employed to approximate unknowns functions in the first-order system. In one case, we use nearly the same DNN to approximate all unknown functions and in the other case, we use totally different DNNs for different unknown functions. Numerous results of MIM with different loss functions and different choices of DNNs are given for four types of PDEs. In most cases, MIM provides better approximations (not only for high-order derivatives of the PDE solution but also for the PDE solution itself) than DGM with nearly the same DNN and the same execution time, sometimes by more than one order of magnitude. MIM with multiple DNNs often provides better approximations than MIM with only one DNN, sometimes by more than one order of magnitude. Numerical results also indicate interesting connections between MIM and classical numerical methods. Therefore, we expect MIM to open up a possibly systematic way to understand and improve deep learning for solving PDEs from the perspective of classical numerical analysis.

A Priori Error Estimate of Deep Mixed Residual Method for Elliptic PDEs

Error Analysis of Deep Ritz Methods for Elliptic Equations.

Error Analysis of the Deep Mixed Residual Method for High-order Elliptic Equations

Convergence Rate Analysis for Deep Ritz Method

Deep Ritz Methods for Laplace Equations with Dirichlet Boundary Condition

Convergence Analysis of a Quasi-Monte Carlo-based Deep Learning Algorithm for Solving Partial Differential Equations

Machine Learning For Elliptic PDEs: Fast Rate Generalization Bound, Neural Scaling Law and Minimax Optimality

Error Analysis of Mixed Residual Methods for Elliptic Equations

Error Analysis of Three-Layer Neural Network Trained with PGD for Deep Ritz Method

A comparison study of deep Galerkin method and deep Ritz method for elliptic problems with different boundary conditions

Deep Mixed Residual Method for Solving Pde-Constrained Optimization Problems

Quasi-Monte Carlo sampling for machine-learning partial differential equations

A Deep Double Ritz Method (D$^2$RM) for solving Partial Differential Equations using Neural Networks

A Mixed Residual Hybrid Method for Failure Probability Estimation

Numerical solution for high-dimensional partial differential equations based on deep learning with residual learning and data-driven learning

Statistical Numerical PDE : Fast Rate, Neural Scaling Law and When it’s Optimal

MIM: A deep mixed residual method for solving high-order partial differential equations

A Priori Estimation of the Approximation, Optimization and Generalization Errors of Random Neural Networks for Solving Partial Differential Equations

Accurate adaptive deep learning method for solving elliptic problems

Deep Finite Volume Method for High-Dimensional Partial Differential Equations

D3M: A deep domain decomposition method for partial differential equations