Abstract:Neural operators effectively solve PDE problems from data without knowing the explicit equations, which learn the map from the input sequences of observed samples to the predicted values. Most existing works build the model in the original geometric space, leading to high computational costs when the number of sample points is large. We present the Latent Neural Operator (LNO) solving PDEs in the latent space. In particular, we first propose Physics-Cross-Attention (PhCA) transforming representation from the geometric space to the latent space, then learn the operator in the latent space, and finally recover the real-world geometric space via the inverse PhCA map. Our model retains flexibility that can decode values in any position not limited to locations defined in the training set, and therefore can naturally perform interpolation and extrapolation tasks particularly useful for inverse problems. Moreover, the proposed LNO improves both prediction accuracy and computational efficiency. Experiments show that LNO reduces the GPU memory by 50%, speeds up training 1.8 times, and reaches state-of-the-art accuracy on four out of six benchmarks for forward problems and a benchmark for inverse problem. Code is available at <a class="link-external link-https" href="https://github.com/L-I-M-I-T/LatentNeuralOperator" rel="external noopener nofollow">this https URL</a>.
What problem does this paper attempt to address?
### Problems the paper attempts to solve
The paper "Latent Neural Operator for Solving Forward and Inverse PDE Problems" attempts to solve the forward and inverse problems of partial differential equations (PDEs). Specifically, the paper proposes a model named Latent Neural Operator (LNO), aiming to improve the accuracy and computational efficiency of solving PDEs by learning PDE operators in the latent space.
### Main contributions
1. **Flexibility**:
- Proposed the Physics - Cross - Attention (PhCA) module, which can decouple the positions of input and output samples and learn the latent space from data.
- Constructed the Latent Neural Operator (LNO) model for solving forward and inverse PDE problems.
2. **Efficiency**:
- The LNO model performs data transformation only once in the encoder and decoder, reducing GPU memory usage, training time, and the number of model parameters.
- Experimental results show that LNO reduces GPU memory by 50% and increases the training speed by 1.8 times.
3. **Accuracy**:
- In the benchmark tests of four forward problems (Pipe, Elasticity, Darcy, Navier - Stokes) and one inverse problem (Burgers' equation), LNO has achieved state - of - the - art accuracy.
### Method overview
1. **Problem definition**:
- Forward problem: Given the differential operator \( \mathcal{L}_a \) of PDE and the boundary/initial condition \( b(x) \), solve the solution \( u(x) \) of PDE.
- Inverse problem: Given partial observation data \( \{(x, u(x))\} \), infer the PDE that generates the observation data and its possible boundary/initial conditions or parameters \( a \).
2. **Latent Neural Operator (LNO) model**:
- **Embedding layer**: Lift the input data to a high - dimensional representation.
- **Encoder**: Use PhCA to transform the input data into a learnable latent space.
- **Self - attention layer**: Learn PDE operators through a series of Transformer blocks in the latent space.
- **Decoder**: Restore the latent representation to the actual geometric space through inverse PhCA.
3. **Core module: Physics - Cross - Attention (PhCA)**:
- The PhCA module transforms the data representation from the actual geometric space to the latent space in the encoder and decoder respectively, and then transforms it back from the latent space to the actual geometric space.
- Different from existing methods, the latent space in LNO is learned from data rather than predefined.
### Experimental results
1. **Forward problems**:
- On six popular benchmark tests (Darcy, NS2d, Airfoil, Elasticity, Plasticity, Pipe), LNO achieved the best accuracy in four benchmark tests, and its performance in the other two benchmark tests is close to SOTA with only half of the computational cost.
2. **Inverse problems**:
- Using Burgers' equation as a test case for inverse problems, LNO outperforms DeepONet and GNOT in both stages, especially in the completion and propagation stages.
3. **Efficiency**:
- Compared with Transolver, LNO has significant improvements in the number of parameters, memory consumption, and training time. Specifically, LNO reduces the number of parameters by 30%, memory consumption by 50%, and accelerates the training speed by 1.8 times.
### Conclusion
The paper proposes a new Latent Neural Operator (LNO) model, which successfully solves the forward and inverse PDE problems by learning PDE operators in the latent space. LNO not only improves the accuracy and computational efficiency of the model but also shows good performance.