Abstract:This paper introduces PDEformer, a neural solver for partial differential equations (PDEs) capable of simultaneously addressing various types of PDEs. We propose to represent the PDE in the form of a computational graph, facilitating the seamless integration of both symbolic and numerical information inherent in a PDE. A graph Transformer and an implicit neural representation (INR) are employed to generate mesh-free predicted solutions. Following pretraining on data exhibiting a certain level of diversity, our model achieves zero-shot accuracies on benchmark datasets that is comparable to those of specifically trained expert models. Additionally, PDEformer demonstrates promising results in the inverse problem of PDE coefficient recovery.
What problem does this paper attempt to address?
The problem that this paper attempts to solve is to develop a fundamental model capable of efficiently handling various partial differential equations (PDEs). Specifically, the paper introduces a neural solver named PDEformer, aiming to solve multiple types of PDEs simultaneously. PDEformer seamlessly integrates the symbolic and numerical information in PDEs by representing PDEs in the form of a computational graph. It uses Graph Transformer and Implicit Neural Representation (INR) to generate mesh - free predicted solutions. After pre - training on data with a certain degree of diversity, the zero - sample accuracy of PDEformer on the benchmark dataset is comparable to that of expert models trained specifically. In addition, PDEformer also shows promising results in the inverse problem of PDE coefficient recovery.
### Main Contributions
1. **Generality**: PDEformer aims to construct a fundamental PDE model with the highest generality, which can ideally handle any PDE. For a new PDE, this model can be directly used for zero - sample inference, or fine - tuned with a small number of solution snapshots.
2. **Computational Graph Representation**: Represent the symbolic form of PDE as a computational graph, ensuring that the graph structure, its node types, and feature vectors encapsulate all the symbolic and numerical information required to solve the PDE.
3. **High Performance**: The pre - trained PDEformer shows higher zero - sample prediction accuracy on the benchmark dataset, and its performance can be further improved after fine - tuning.
4. **Inverse Problem Application**: PDEformer performs well in the inverse problem of PDE coefficient recovery, verifying its potential application value in downstream tasks.
### Method Overview
The method of PDEformer mainly consists of the following steps:
1. **Computational Graph Construction**: Represent the symbolic information of PDE as a computational graph, where nodes can represent unknown field variables, scalar coefficients, initial conditions, differential operations, etc., and edges represent operands in operations.
2. **Graph Data Encoding**: Use Graph Transformer to integrate the symbolic and numerical information in the computational graph into a latent code.
3. **Decoding PDE Solutions**: Use INR to generate mesh - free predicted solutions according to coordinate inputs.
### Experimental Results
1. **Pre - training Stage**: Generate a dataset containing 500,000 samples, covering different types of PDEs, coefficients, and initial conditions. After pre - training, the relative L2 error of PDEformer on the training set is 0.0104, and on the test set is 0.0128.
2. **Forward Problem**: Evaluate the forward problem performance of PDEformer on the PDEBench dataset, including Burgers, Advection, and 1D Reaction - Diffusion PDEs. The results show that PDEformer performs excellently in zero - sample inference, even surpassing the baseline models trained specifically for these datasets.
3. **Inverse Problem**: Use PDEformer to conduct the inverse problem of PDE coefficient recovery. Even at a high noise level, it can effectively recover most PDE coefficients.
### Conclusion
By representing PDEs as computational graphs and combining Graph Transformer and INR, PDEformer successfully constructs a PDE solver with strong generality and high performance. This model not only performs well in forward problems but also shows strong application potential in inverse problems. Although the current experiments are limited to one - dimensional PDEs, this achievement lays an important foundation for constructing more widely applicable fundamental PDE models.