Joost A. A. Opschoor,Philipp C. Petersen,Christoph Schwab
Abstract:We introduce a conceptual framework for numerically solving linear elliptic, parabolic, and hyperbolic PDEs on bounded, polytopal domains in euclidean spaces by deep neural networks. The PDEs are recast as minimization of a least-squares (LSQ for short) residual of an equivalent, well-posed first-order system, over parametric families of deep neural networks. The associated LSQ residual is a) equal or proportional to a weak residual of the PDE, b) additive in terms of contributions from localized subnetworks, indicating locally ``out-of-equilibrium'' of neural networks with respect to the PDE residual, c) serves as numerical loss function for neural network training, and d) constitutes, even with incomplete training, a computable, (quasi-)optimal numerical error estimator in the context of adaptive LSQ finite element methods. In addition, an adaptive neural network growth strategy is proposed which, assuming exact numerical minimization of the LSQ loss functional, yields sequences of neural networks with realizations that converge rate-optimally to the exact solution of the first order system LSQ formulation.
What problem does this paper attempt to address?
The problem that this paper attempts to solve is: how to use deep neural networks (DNN) to numerically solve linear elliptic, parabolic and hyperbolic partial differential equations (PDEs), especially in Euclidean spaces within bounded polyhedral domains. Specifically, the author proposes a conceptual framework based on the First - order System Least - Squares (FoSLS). By recasting the PDE as an equivalent least - squares residual problem of a first - order system and minimizing it over a family of parameterized deep neural networks.
### Specific Problem Description
1. **Numerical Solution of PDEs**:
- The author hopes to use deep neural networks to numerically solve linear elliptic, parabolic and hyperbolic PDEs.
- These PDEs are defined in Euclidean spaces within bounded polyhedral domains.
2. **Least - Squares Residual Method**:
- Recast the PDE as an equivalent least - squares residual problem of a first - order system.
- The minimization objective is to minimize the least - squares residual over a family of parameterized deep neural networks.
3. **Loss Function Design**:
- Design a computable loss function. This loss function is based on the least - squares residual and can be used as a numerical loss function for neural network training.
- Even when the network training is incomplete, this loss function can be used as a (quasi) - optimal numerical error estimator in the adaptive least - squares finite - element method.
4. **Adaptive Neural Network Growth Strategy**:
- Propose an adaptive neural network growth strategy. Assuming the exact numerical minimization of the least - squares loss function, the generated neural network sequence converges to the exact solution of the first - order system least - squares formula at an optimal rate.
### Mathematical Expressions
- **Least - Squares Residual**:
\[
E_{\text{FoSLS}}(\theta)=\|F - LU_{\theta}\|_{L(D)}
\]
where \(U_{\theta}\) is the function implemented by the neural network, \(L\) is the differential operator, \(F\) is the given data, and \(L(D)\) is an appropriate Hilbert space.
- **Error Representation**:
\[
0\leq E(\theta):=\|U - U_{\theta}\|_{V(D)}=\|F - LU_{\theta}\|_{L(D)}, \quad \theta\in\Theta
\]
where \(V(D)\) is an appropriate Sobolev space.
- **Optimization Condition**:
\[
U = \arg\min_{v\in V(D)} LS(v; F), \quad \text{with}\quad LS(v; F):=\|F - Lv\|_{L(D)}^2
\]
### Key Contributions
1. **Structure - Preserving Neural Network Design**:
- The design of the neural network is structure - preserving, that is, they must be de Rham - compatible on the shrinking spatial domain \(D\) or the space - time domain \(D=(0, T)\times G\).
2. **Transport Principle**:
- Use de Rham - compatible finite - element spaces as feature spaces, so that known mathematical results can be directly applied to FoSLS NN approximation analysis.
3. **Physically Correct Computable Loss Function**:
- For a wide range of PDE boundary - value and initial - boundary - value problems, provide a unique minimizer, thus obtaining a computable numerical loss function based on the \(L^2(D)\)-norm.
4. **High - Order Numerical Integration**:
- Utilize standard high - order numerical integration methods, so that the least - squares residuals of the \(L^2(D)\)-basis can be accessed numerically.
5. **Localized Loss Function**:
- The loss function consists of contributions from local sub - networks, which correspond to sub - domains of the physical domain \(D\).
6. **Convergence and Adaptive Growth Strategy**:
- Propose an adaptive neural network growth strategy to ensure that the generated neural network sequence converges to the minimizer of the least - squares functional at an optimal rate.
In summary