Seismic Foundation Model (SFM): a new generation deep learning model in geophysics

Hanlin Sheng,Xinming Wu,Xu Si,Jintao Li,Sibo Zhang,Xudong Duan

2023-12-15

Abstract:While computer science has seen remarkable advancements in foundation models, which remain underexplored in geoscience. Addressing this gap, we introduce a workflow to develop geophysical foundation models, including data preparation, model pre-training, and adaption to downstream tasks. From 192 globally collected 3-D seismic volumes, we create a carefully curated dataset of 2,286,422 2-D seismic images. Fully using these unlabeled images, we employ the self-supervised learning to pre-train a Transformer-based Seismic Foundation Model (SFM) for producing all-purpose seismic features that work across various tasks and surveys. Through experiments on seismic facies classification, geobody identification, interpolation, denoising, and inversion, our pre-trained model demonstrates versatility, generalization, scalability, and superior performance over baseline models. Conclusively, we provide a foundation model and vast dataset to advance AI in geophysics, addressing challenges (poor generalization, lacking labels, and repetitive training for task-specified models) of applying AI in geophysics and paving the way for future innovations in geoscience.

Geophysics

What problem does this paper attempt to address?

### Problems Addressed by the Paper

The paper aims to address several major challenges faced when applying artificial intelligence in geophysics, including poor model generalization, lack of labeled data, and the issue of repeatedly training models for specific tasks. Specifically, the authors propose a new generation of deep learning model based on self-supervised learning—the Seismic Foundation Model (SFM)—to tackle these challenges.

#### Main Contributions
1. **Dataset Construction**: Collected 192 3-D seismic datasets globally and carefully selected 2,286,422 2-D seismic images from them to construct a large-scale and diverse unlabeled dataset.
2. **Self-Supervised Pretraining**: Utilized these unlabeled seismic images to pretrain the SFM using a generative self-supervised learning strategy and Transformer architecture, thereby generating general seismic features suitable for various tasks.
3. **Downstream Task Validation**: Demonstrated the generalization ability and superior performance of the SFM by validating the pretrained model's performance on multiple downstream tasks, including seismic facies classification, geological body identification, interpolation, denoising, and inversion.

### Summary
This paper proposes a new Seismic Foundation Model (SFM) that addresses the limitations of traditional deep learning models in geophysics through self-supervised learning and pretraining on a large-scale dataset, providing a solid foundation for future innovations in the geoscience field.