Meta-PINN: Meta learning for improved neural network wavefield solutions

Shijun Cheng,Tariq Alkhalifah

2024-01-21

Abstract:Physics-informed neural networks (PINNs) provide a flexible and effective alternative for estimating seismic wavefield solutions due to their typical mesh-free and unsupervised features. However, their accuracy and training cost restrict their applicability. To address these issues, we propose a novel initialization for PINNs based on meta learning to enhance their performance. In our framework, we first utilize meta learning to train a common network initialization for a distribution of medium parameters (i.e. velocity models). This phase employs a unique training data container, comprising a support set and a query set. We use a dual-loop approach, optimizing network parameters through a bidirectional gradient update from the support set to the query set. Following this, we use the meta-trained PINN model as the initial model for a regular PINN training for a new velocity model in which the optimization of the network is jointly constrained by the physical and regularization losses. Numerical results demonstrate that, compared to the vanilla PINN with random initialization, our method achieves a much fast convergence speed, and also, obtains a significant improvement in the results accuracy. Meanwhile, we showcase that our method can be integrated with existing optimal techniques to further enhance its performance.

Geophysics

What problem does this paper attempt to address?

This paper proposes a novel initialization method based on meta-learning to improve the accuracy and training cost of Physics-Informed Neural Networks (PINNs) in estimating seismic wavefield solutions. PINNs provide a flexible and efficient approach for estimating seismic wavefields due to their grid-free and unsupervised nature, but their accuracy and training cost limit their practicality. To address these issues, the paper introduces a meta-learning-based initialization method to enhance the performance of PINNs. This method trains a general network initialization through meta-learning, which is applicable to a range of distributions of medium parameters (i.e., velocity models). By using unique training data containers with support and query sets, the network parameters are optimized through a bi-level optimization process. Subsequently, the meta-trained PINN model is used as the initial model for conventional PINN training with the new velocity model, where the network optimization is constrained by the joint losses of physical and regularization terms. Numerical results demonstrate that compared to randomly initialized PINNs, this method achieves faster convergence and significantly improves result accuracy. Furthermore, this method can be combined with existing state-of-the-art techniques to further enhance performance.

The main contribution of the paper is the introduction of meta-learning to optimize the initialization of PINNs, improving their efficiency and accuracy in simulating seismic wavefields, especially for adapting to different velocity models. Through this approach, PINN is able to converge to accurate wavefield solutions faster, reducing the computational cost for large-scale problems.