LatentPINNs: Generative physics-informed neural networks via a latent representation learning

Mohammad H. Taufik,Tariq Alkhalifah

2023-05-12

Abstract:Physics-informed neural networks (PINNs) are promising to replace conventional partial differential equation (PDE) solvers by offering more accurate and flexible PDE solutions. However, they are hampered by the relatively slow convergence and the need to perform additional, potentially expensive, training for different PDE parameters. To solve this limitation, we introduce latentPINN, a framework that utilizes latent representations of the PDE parameters as additional (to the coordinates) inputs into PINNs and allows for training over the distribution of these parameters. Motivated by the recent progress on generative models, we promote the use of latent diffusion models to learn compressed latent representations of the PDE parameters distribution and act as input parameters to NN functional solutions. We use a two-stage training scheme in which the first stage, we learn the latent representations for the distribution of PDE parameters. In the second stage, we train a physics-informed neural network over inputs given by randomly drawn samples from the coordinate space within the solution domain and samples from the learned latent representation of the PDE parameters. We test the approach on a class of level set equations given by the nonlinear Eikonal equation. We specifically share results corresponding to three different sets of Eikonal parameters (velocity models). The proposed method performs well on new phase velocity models without the need for any additional training.

Machine Learning,Computational Physics

What problem does this paper attempt to address?

The paper attempts to address two main issues with Physics-Informed Neural Networks (PINNs) when solving Partial Differential Equations (PDEs):

1. **Slow convergence speed**: Randomly initialized PINNs require thousands of iterations to converge, which significantly increases the cost of obtaining PDE solutions.
2. **Additional training for different PDE parameters**: Existing PINNs methods are trained for specific PDE parameters, and when faced with new PDE parameters, the model usually needs to be retrained or fine-tuned.

To solve these problems, the authors propose the latentPINN framework, which uses latent variables to represent PDE parameters and passes them as inputs to PINNs. In this way, the trained PINNs can be viewed as a generative model capable of generating solutions under different PDE parameters based on the latent variables, without the need for additional training for each new parameter. Specifically, the authors use a two-stage training scheme: first learning a compressed latent representation of the PDE parameters; then training a PINN model with latent variable inputs on this basis.

The method was tested on the nonlinear Eikonal equation and demonstrated the ability to achieve good performance when handling new phase velocity models without additional training.