Abstract:Initial boundary value problems arise commonly in applications with engineering and natural systems governed by nonlinear partial differential equations (PDEs). Operator learning is an emerging field for solving these equations by using a neural network to learn a map between infinite dimensional input and output function spaces. These neural operators are trained using a combination of data (observations or simulations) and PDE-residuals (physics-loss). A major drawback of existing neural approaches is the requirement to retrain with new initial/boundary conditions, and the necessity for a large amount of simulation data for training. We develop a physics-informed transformer neural operator (named PINTO) that efficiently generalizes to unseen initial and boundary conditions, trained in a simulation-free setting using only physics loss. The main innovation lies in our new iterative kernel integral operator units, implemented using cross-attention, to transform the PDE solution's domain points into an initial/boundary condition-aware representation vector, enabling efficient learning of the solution function for new scenarios. The PINTO architecture is applied to simulate the solutions of important equations used in engineering applications: advection, Burgers, and steady and unsteady Navier-Stokes equations (three flow scenarios). For these five test cases, we show that the relative errors during testing under challenging conditions of unseen initial/boundary conditions are only one-fifth to one-third of other leading physics informed operator learning methods. Moreover, our PINTO model is able to accurately solve the advection and Burgers equations at time steps that are not included in the training collocation points. The code is available at $\texttt{<a class="link-external link-https" href="https://github.com/quest-lab-iisc/PINTO" rel="external noopener nofollow">this https URL</a>}$
What problem does this paper attempt to address?
The problem that this paper attempts to solve is: how to use neural networks to efficiently learn and predict the generalized solutions of initial - boundary value problems (IBVP) described by nonlinear partial differential equations (PDEs), especially training the model without simulation data and being able to generalize to unseen initial and boundary conditions.
### Detailed Explanation:
1. **Background and Problem**
- Initial - boundary value problems (IBVP) are very common in engineering and natural systems, and these systems are usually described by nonlinear partial differential equations (PDEs).
- Although traditional numerical methods (such as finite difference, finite element, etc.) can solve these problems, they need to be re - solved every time new initial or boundary conditions are encountered, consuming a large amount of computing resources.
- Existing neural network methods (such as PINN) can learn the solutions of PDEs in some cases, but usually require a large amount of simulation data for training and are difficult to generalize to unseen initial and boundary conditions.
2. **Core Contributions of the Paper**
- A new model named PINTO (Physics - Informed Transformer Neural Operator) has been developed, which can be trained only through physical loss (physics loss) without using any simulation data.
- PINTO introduces a new cross - attention mechanism (cross - attention unit), enabling the model to combine the coordinates of the space - time domain with the initial / boundary conditions, so as to effectively learn the solutions of PDEs and generalize to new initial and boundary conditions.
- The experimental results show that PINTO performs excellently in several important PDE test cases (such as the convection equation, Burgers' equation, Navier - Stokes equation, etc.), and the relative error is only one - fifth to one - third of other leading methods.
3. **Specific Implementation**
- **Mathematical Problem Formulation**
Consider a PDE in general form:
\[
N(h, X; \alpha)=f \quad \text{in} \quad \Omega, \quad B(h, X_b)=b \quad \text{on} \quad \partial\Omega
\]
where \( h \) is the solution field, \( N \) is the nonlinear differential operator, \( X \) is the space - time domain coordinate, \( \alpha \) is the parameter vector of the PDE, \( f \) is the source term, \( B \) is the initial / boundary operator, \( X_b \) is the boundary coordinate, and \( b \) is the initial / boundary condition.
- **Neural Operator Definition and Loss Function**
The goal of PINTO is to learn a mapping \( G: A \to H \) from the initial / boundary conditions to the solution field and train by minimizing the physical loss and the boundary condition loss:
\[
\min_{\Theta} \sum_{k = 1}^{K}\left(\lambda_1 \sum_{j = 1}^{N_c}|f_{c,k}-N(G(\Theta; X_{c,k}); \alpha)|^2+\lambda_2 \sum_{j = 1}^{N_{ib}}|b_{ib,k}-B(G(\Theta; X_{ib,k}))|^2\right)
\]
- **Cross - Attention Mechanism**
The key innovation of PINTO is the introduction of the cross - attention mechanism, which combines the representation of the query points with the initial / boundary conditions through the iterative kernel integral operator, thereby improving the generalization ability of the model.
4. **Application and Verification**
- PINTO has been verified in several classic PDE test cases, including 1D convection equation, 1D nonlinear Burgers' equation, 2D unsteady Beltrami flow, 2D steady Kovasznay flow and 2D steady cavity - driven flow.
- The experimental results show...