Abstract:Estimating acoustic impedance from seismic data is a crucial step in reservoir characterization. While data-driven impedance inversion based on deep learning has shown promising results, it relies heavily on extensive well logs for labeling, which is often impractical in many exploration scenarios. Recently, the zero-shot or few-shot learning performance of Pretrained Foundation Models like Generative Pre-trained Transformer (GPT) and Mask Auto Encoder (MAE) has highlighted that knowledge learned from vast amounts of unlabeled data can be transferred to downstream tasks with minimal labeled data. However, applying Transformer-based representation learning models to 3D seismic data inversion poses three challenges: (1) Computational and memory constraints due to the high-dimensional nature of the data; (2) Difficulty in extracting fine-grained image features using Transformers, hampering high-frequency impedance inversion; (3) Fixed input size in Transformers, leading to inversion artifacts. In this work, we introduce the Seismic Mask Auto Encoder (SeisMAE), a Transformer-based representation model tailored for the inversion of 3D seismic data. It incorporates three key components: (1) Aggregated dimensionality reduction encoding to handle redundancy in seismic data, significantly improving computational efficiency; (2) Multi-scale self-attention feature fusion to enhance the model's capacity for low-level feature representation; and (3) A stitching decoding strategy to eliminate inversion stitching artifacts. Experimental validations highlight the efficacy of our approach. On the synthetic SEAM I dataset, we demonstrate the effectiveness of each component and SeisMAE's superior performance. For real-world data on The Netherlands F3, SeisMAE delivers reliable inversion outcomes with only four labeled examples. We compared SeisMAE against various inversion techniques, including 1D Convolutional Neural Network (1D-CNN), UNet-based, HRNet-based, and TransInver, where SeisMAE exhibited significant advantages.

QuakeFormer: A Uniform Approach to Earthquake Ground Motion Prediction Using Masked Transformers

Nowcasting Earthquakes with QuakeGPT: Methods and First Results

Rapid Earthquake Magnitude Estimation Using Deep Learning.

Time Series Foundation Models and Deep Learning Architectures for Earthquake Temporal and Spatial Nowcasting

Earthquake magnitude and location estimation from real time seismic waveforms with a transformer network

SeisT: A Foundational Deep-Learning Model for Earthquake Monitoring Tasks

SeisT: A foundational deep learning model for earthquake monitoring tasks

A Global Earthquake Prediction Model Based on Spherical Convolutional LSTM

SeisGPT: A Physics-Informed Data-Driven Large Model for Real-Time Seismic Response Prediction

WaveCastNet: An AI-enabled Wavefield Forecasting Framework for Earthquake Early Warning

The transformer earthquake alerting model: a new versatile approach to earthquake early warning

An All‐In‐One Rapid Prediction of Ground Motion Intensity Measures Hybrid Network for Multi‐Task in the North‐South Seismic Belt of China

Transformer based neural network for daily ground settlement prediction of foundation pit considering spatial correlation

Earthquake transformer—an attentive deep-learning model for simultaneous earthquake detection and phase picking

QuakeFlow: a scalable machine-learning-based earthquake monitoring workflow with cloud computing

3D seismic mask auto encoder: Seismic inversion using transformer-based reconstruction representation learning

Learning Physics for Unveiling Hidden Earthquake Ground Motions via Conditional Generative Modeling

Deep Learning Peak Ground Acceleration Prediction Using Single-Station Waveforms

TransFrameNet: A Transformer-Based Approach for Generalized Seismic Performance Prediction of Building Structures

Earthquake Nowcasting with Deep Learning

Seismic Attributes Aided Horizon Interpretation Using an Ensemble Dense Inception Transformer Network