When Deep Learning Meets Geophysics

Jianwei Ma,Siwei Yu

DOI: https://doi.org/10.1029/2021eo162802

2021-01-01

Eos

Abstract:Traditional physical models are no longer the only foundational tools for processing geophysical data; "big data" help to reveal the laws of geophysics from new angles with exciting results so far.

What problem does this paper attempt to address?

Deep Learning For Geophysics: Current And Future Trends

Siwei Yu,Jianwei Ma

DOI: https://doi.org/10.1029/2021RG000742

2021-01-01

Reviews of Geophysics

Abstract:Recently deep learning (DL), as a new data-driven technique compared to conventional approaches, has attracted increasing attention in geophysical community, resulting in many opportunities and challenges. DL was proven to have the potential to predict complex system states accurately and relieve the "curse of dimensionality" in large temporal and spatial geophysical applications. We address the basic concepts, state-of-the-art literature, and future trends by reviewing DL approaches in various geosciences scenarios. Exploration geophysics, earthquakes, and remote sensing are the main focuses. More applications, including Earth structure, water resources, atmospheric science, and space science, are also reviewed. Additionally, the difficulties of applying DL in the geophysical community are discussed. The trends of DL in geophysics in recent years are analyzed. Several promising directions are provided for future research involving DL in geophysics, such as unsupervised learning, transfer learning, multimodal DL, federated learning, uncertainty estimation, and active learning. A coding tutorial and a summary of tips for rapidly exploring DL are presented for beginners and interested readers of geophysics.
Finite Element Geotechnical Analysis Incorporating Deep Learning-Based Soil Model

Q. Z. Guan,Z. X. Yang,N. Guo,Z. Hu

DOI: https://doi.org/10.1016/j.compgeo.2022.105120

IF: 5.218

2023-01-01

Computers and Geotechnics

Abstract:Owing to the complicated mechanical behaviors of soils, their constitutive models often involve obscureformulations and suffer from poor applicability in engineering practice. In this study, a novel framework for the finite element (FE) analysis of geotechnical engineering problems is proposed, in which a deep learning (DL) model is employed to depict the constitutive behaviors of soils, circumventing the difficulties associated with conventional approaches. The DL model can incorporate different neural network architectures and is trained with stress–strain data, obtained either experimentally or numerically, before being integrated into the FE solver for analyzing various boundary value problems (BVPs). During the FE solution, the DL model receives strains at the Gauss integration points and returns the predicted stresses to advance the computation. The applicability and capacity of the framework were demonstrated by analyzing three BVPs, in which different geometries, meshes, and boundary conditions were considered. It was shown that the framework is capable of reproducing satisfactory solutions without resorting to any constitutive theory. Furthermore, the use of the DL model not only avoids the stress integration of the conventional FE analysis, but also leads to better computational efficiency.
Data-driven geophysics: from dictionary learning to deep learning

Siwei Yu,Jianwei Ma

DOI: https://doi.org/10.48550/arXiv.2007.06183

IF: 3.264

2020-07-13

Geophysics

Abstract:Understanding the principles of geophysical phenomena is an essential and challenging task. "Model-driven" approaches have supported the development of geophysics for a long time; however, such methods suffer from the curse of dimensionality and may inaccurately model the subsurface. "Data-driven" techniques may overcome these issues with increasingly available geophysical data. In this article, we review the basic concepts of and recent advances in data-driven approaches from dictionary learning to deep learning in a variety of geophysical scenarios. Explorational geophysics including data processing, inversion and interpretation will be mainly focused. Artificial intelligence applications on geoscience involving deep Earth, earthquake, water resource, atmospheric science, satellite remoe sensing and space sciences are also reviewed. We present a coding tutorial and a summary of tips for beginners and interested geophysical readers to rapidly explore deep learning. Some promising directions are provided for future research involving deep learning in geophysics, such as unsupervised learning, transfer learning, multimodal deep learning, federated learning, uncertainty estimation, and activate learning.
Deep Learning for Gravity and Magnetic Data Interpolation

Feng Wang,Shengchang Chen,Yanan Liu

DOI: https://doi.org/10.1190/segam2019-3216102.1

2019-01-01

Abstract:Deep learning has achieved a great success in various fields in recent years. As a powerful tool, deep learning is used to interpolate the gravity and magnetic data with block lack of data in this paper. We firstly designed a deep convolution neural network (CNN), and mask dataset was created to simulate the block lack in the data. The network was trained by using the earth’s gravity anomalies dataset (bouguer, isostatic and surface free-air) and the earth magnetic anomaly dataset. We demonstrated the feasibility of this approach by numerical experiments, which indicated that this method is able to effectively recover the gravity and magnetic data with block lack. The experiments also demonstrated that CNN gains competitive performance compared with traditional interpolation methods. Presentation Date: Wednesday, September 18, 2019 Session Start Time: 9:20 AM Presentation Time: 10:35 AM Location: Poster Station 1 Presentation Type: Poster
Deep learning for geological hazards analysis: Data, models, applications, and opportunities

Zhengjing Ma,Gang Mei

DOI: https://doi.org/10.1016/j.earscirev.2021.103858

2021-12-01

Abstract:As natural disasters are induced by geodynamic activities or abnormal changes in the environment, geological hazards tend to wreak havoc on the environment and human society. Recently, the dramatic increase in the volume of various types of Earth observation ‘big data’ from multiple sources, and the rapid development of deep learning as a state-of-the-art data analysis tool, have enabled novel advances in geological hazard analysis, with the ultimate aim to mitigate the devastation associated with these hazards. Motivated by numerous applications, this paper presents an overview of the advances in the utilization of deep learning for geological hazard analysis. First, six commonly available Earth observation data sources are described, e.g., unmanned aerial vehicles, satellite platforms, and in-situ monitoring systems. Second, the deep learning background and six typical deep learning models are introduced, such as convolutional neural networks and recurrent neural networks. Third, focusing on six typical geological hazards, i.e., landslides, debris flows, rockfalls, avalanches, earthquakes, and volcanoes, the deep learning applications for geological hazard analysis are reviewed, and common application paradigms are summarized. Finally, the challenges and opportunities for the application of deep learning models for geological hazard analysis are highlighted, with the aim to inspire further related research.

geosciences, multidisciplinary
Improving AI System Awareness of Geoscience Knowledge: Symbiotic Integration of Physical Approaches and Deep Learning

Shijie Jiang,Yi Zheng,Dimitri Solomatine

DOI: https://doi.org/10.1029/2020GL088229

IF: 5.2

2020-01-01

Geophysical Research Letters

Abstract:Modeling dynamic geophysical phenomena is at the core of Earth and environmental studies. The geoscientific community relying mainly on physical representations may want to consider much deeper adoption of artificial intelligence (AI) instruments in the context of AI's global success and emergence of big Earth data. A new perspective of using hybrid physics-AI approaches is a grand vision, but actualizing such approaches remains an open question in geoscience. This study develops a general approach to improving AI geoscientific awareness, wherein physical approaches such as temporal dynamic geoscientific models are included as special recurrent neural layers in a deep learning architecture. The illustrative case of runoff modeling across the conterminous United States demonstrates that the physics-aware DL model has enhanced prediction accuracy, robust transferability, and good intelligence for inferring unobserved processes. This study represents a firm step toward realizing the vision of tackling Earth system challenges by physics-AI integration.
Deep Learning for Spatiotemporal Big Data: A Vision on Opportunities and Challenges

Zhe Jiang

2023-10-31

Abstract:With advancements in GPS, remote sensing, and computational simulation, an enormous volume of spatiotemporal data is being collected at an increasing speed from various application domains, spanning Earth sciences, agriculture, smart cities, and public safety. Such emerging geospatial and spatiotemporal big data, coupled with recent advances in deep learning technologies, foster new opportunities to solve problems that have not been possible before. For instance, remote sensing researchers can potentially train a foundation model using Earth imagery big data for numerous land cover and land use modeling tasks. Coastal modelers can train AI surrogates to speed up numerical simulations. However, the distinctive characteristics of spatiotemporal big data pose new challenges for deep learning technologies. This vision paper introduces various types of spatiotemporal big data, discusses new research opportunities in the realm of deep learning applied to spatiotemporal big data, lists the unique challenges, and identifies several future research needs.

Machine Learning,Artificial Intelligence,Computer Vision and Pattern Recognition
Deep learning and its application in geochemical mapping

Renguang Zuo,Yihui Xiong,Jian Wang,Emmanuel John M. Carranza

DOI: https://doi.org/10.1016/j.earscirev.2019.02.023

2019-05-01

Abstract:Machine learning algorithms have been applied widely in the fields of natural science, social science and engineering. It can be expected that machine learning approaches especially deep learning algorithms will help geoscientists to discover mineral deposits through processing of various geoscience datasets. This study reviews the state-of-the-art application of deep learning algorithms for processing geochemical exploration data and mining the geochemical patterns. Deep learning algorithms can deal with complex and nonlinear problems and, therefore, can enhance the identification of geochemical anomalies and the recognition of hidden patterns. Applied geochemistry needs more applications of machine learning and/or deep learning algorithms.

geosciences, multidisciplinary
Deep-learning seismology

S. Mostafa Mousavi,Gregory C. Beroza

DOI: https://doi.org/10.1126/science.abm4470

IF: 56.9

2022-08-12

Science

Abstract:Seismic waves from earthquakes and other sources are used to infer the structure and properties of Earth's interior. The availability of large-scale seismic datasets and the suitability of deep-learning techniques for seismic data processing have pushed deep learning to the forefront of fundamental, long-standing research investigations in seismology. However, some aspects of applying deep learning to seismology are likely to prove instructive for the geosciences, and perhaps other research areas more broadly. Deep learning is a powerful approach, but there are subtleties and nuances in its application. We present a systematic overview of trends, challenges, and opportunities in applications of deep-learning methods in seismology.

multidisciplinary sciences
Deep learning and process understanding for data-driven Earth system science

Markus Reichstein,Gustau Camps-Valls,Bjorn Stevens,Martin Jung,Joachim Denzler,Nuno Carvalhais,Prabhat

DOI: https://doi.org/10.1038/s41586-019-0912-1

IF: 64.8

2019-02-01

Nature

Abstract:Machine learning approaches are increasingly used to extract patterns and insights from the ever-increasing stream of geospatial data, but current approaches may not be optimal when system behaviour is dominated by spatial or temporal context. Here, rather than amending classical machine learning, we argue that these contextual cues should be used as part of deep learning (an approach that is able to extract spatio-temporal features automatically) to gain further process understanding of Earth system science problems, improving the predictive ability of seasonal forecasting and modelling of long-range spatial connections across multiple timescales, for example. The next step will be a hybrid modelling approach, coupling physical process models with the versatility of data-driven machine learning.

multidisciplinary sciences
Seismic Foundation Model (SFM): a new generation deep learning model in geophysics

Hanlin Sheng,Xinming Wu,Xu Si,Jintao Li,Sibo Zhang,Xudong Duan

2023-12-15

Abstract:While computer science has seen remarkable advancements in foundation models, which remain underexplored in geoscience. Addressing this gap, we introduce a workflow to develop geophysical foundation models, including data preparation, model pre-training, and adaption to downstream tasks. From 192 globally collected 3-D seismic volumes, we create a carefully curated dataset of 2,286,422 2-D seismic images. Fully using these unlabeled images, we employ the self-supervised learning to pre-train a Transformer-based Seismic Foundation Model (SFM) for producing all-purpose seismic features that work across various tasks and surveys. Through experiments on seismic facies classification, geobody identification, interpolation, denoising, and inversion, our pre-trained model demonstrates versatility, generalization, scalability, and superior performance over baseline models. Conclusively, we provide a foundation model and vast dataset to advance AI in geophysics, addressing challenges (poor generalization, lacking labels, and repetitive training for task-specified models) of applying AI in geophysics and paving the way for future innovations in geoscience.

Geophysics
Including Physics in Deep Learning -- An example from 4D seismic pressure saturation inversion

Jesper Sören Dramsch,Gustavo Corte,Hamed Amini,Colin MacBeth,Mikael Lüthje

DOI: https://doi.org/10.3997/2214-4609.201901967

2019-04-04

Abstract:Geoscience data often have to rely on strong priors in the face of uncertainty. Additionally, we often try to detect or model anomalous sparse data that can appear as an outlier in machine learning models. These are classic examples of imbalanced learning. Approaching these problems can benefit from including prior information from physics models or transforming data to a beneficial domain. We show an example of including physical information in the architecture of a neural network as prior information. We go on to present noise injection at training time to successfully transfer the network from synthetic data to field data.

Geophysics,Machine Learning,Data Analysis, Statistics and Probability
A 3D Geological Modeling Method Based on Geophysical Data

Xiangyu Yu,Yixian Xu

DOI: https://doi.org/10.3799/dqkx.2015.033

2015-01-01

Earth Science-Journal of China University of Geosciences

Abstract:Inadequate geological sampling information is a major constraint for 3D geological modeling of the deep geological body.Since it is relatively easier to acquire the deep geophysical data,the visualization model of the geophysical data can be formed effectively.In this study,a 3D geological modeling method by the geophysical data collected in the geological survey is proposed.It involves firstly extracting the mapping function from geophysical data to the geological property by geophysical ex-periment on the samples,and then integrating the visualization models constructed from multi-source geophysical data for inter-pretation,and finally converting the interpreted visualization model to the 3D geological model.The method proves feasible for solutions on 3D modeling concerning deep geological bodies in the project.
A Physics-guided Generative AI Toolkit for Geophysical Monitoring

Junhuan Yang,Hanchen Wang,Yi Sheng,Youzuo Lin,Lei Yang

2024-01-06

Abstract:Full-waveform inversion (FWI) plays a vital role in geoscience to explore the subsurface. It utilizes the seismic wave to image the subsurface velocity map. As the machine learning (ML) technique evolves, the data-driven approaches using ML for FWI tasks have emerged, offering enhanced accuracy and reduced computational cost compared to traditional physics-based methods. However, a common challenge in geoscience, the unprivileged data, severely limits ML effectiveness. The issue becomes even worse during model pruning, a step essential in geoscience due to environmental complexities. To tackle this, we introduce the EdGeo toolkit, which employs a diffusion-based model guided by physics principles to generate high-fidelity velocity maps. The toolkit uses the acoustic wave equation to generate corresponding seismic waveform data, facilitating the fine-tuning of pruned ML models. Our results demonstrate significant improvements in SSIM scores and reduction in both MAE and MSE across various pruning ratios. Notably, the ML model fine-tuned using data generated by EdGeo yields superior quality of velocity maps, especially in representing unprivileged features, outperforming other existing methods.

Machine Learning,Artificial Intelligence,Computer Vision and Pattern Recognition,Signal Processing,Geophysics
Artificial intelligence for geoscience: Progress, challenges, and perspectives

Tianjie Zhao,Sheng Wang,Chaojun Ouyang,Min Chen,Chenying Liu,Jin Zhang,Long Yu,Fei Wang,Yong Xie,Jun Li,Fang Wang,Sabine Grunwald,Bryan M Wong,Fan Zhang,Zhen Qian,Yongjun Xu,Chengqing Yu,Wei Han,Tao Sun,Zezhi Shao,Tangwen Qian,Zhao Chen,Jiangyuan Zeng,Huai Zhang,Husi Letu,Bing Zhang,Li Wang,Lei Luo,Chong Shi,Hongjun Su,Hongsheng Zhang,Shuai Yin,Ni Huang,Wei Zhao,Nan Li,Chaolei Zheng,Yang Zhou,Changping Huang,Defeng Feng,Qingsong Xu,Yan Wu,Danfeng Hong,Zhenyu Wang,Yinyi Lin,Tangtang Zhang,Prashant Kumar,Antonio Plaza,Jocelyn Chanussot,Jiabao Zhang,Jiancheng Shi,Lizhe Wang

DOI: https://doi.org/10.1016/j.xinn.2024.100691

2024-08-22

Abstract:This paper explores the evolution of geoscientific inquiry, tracing the progression from traditional physics-based models to modern data-driven approaches facilitated by significant advancements in artificial intelligence (AI) and data collection techniques. Traditional models, which are grounded in physical and numerical frameworks, provide robust explanations by explicitly reconstructing underlying physical processes. However, their limitations in comprehensively capturing Earth's complexities and uncertainties pose challenges in optimization and real-world applicability. In contrast, contemporary data-driven models, particularly those utilizing machine learning (ML) and deep learning (DL), leverage extensive geoscience data to glean insights without requiring exhaustive theoretical knowledge. ML techniques have shown promise in addressing Earth science-related questions. Nevertheless, challenges such as data scarcity, computational demands, data privacy concerns, and the "black-box" nature of AI models hinder their seamless integration into geoscience. The integration of physics-based and data-driven methodologies into hybrid models presents an alternative paradigm. These models, which incorporate domain knowledge to guide AI methodologies, demonstrate enhanced efficiency and performance with reduced training data requirements. This review provides a comprehensive overview of geoscientific research paradigms, emphasizing untapped opportunities at the intersection of advanced AI techniques and geoscience. It examines major methodologies, showcases advances in large-scale models, and discusses the challenges and prospects that will shape the future landscape of AI in geoscience. The paper outlines a dynamic field ripe with possibilities, poised to unlock new understandings of Earth's complexities and further advance geoscience exploration.
Deep Learning Model And Its Application In Big Data

Yuanming Zhou,Shifeng Zhao,Xuesong Wang,Wei Liu

DOI: https://doi.org/10.1007/978-3-319-91797-9_55

2018-01-01

Abstract:In the era of big data, many of the data that previously seemed hard to collect and use began to be utilized, resulting in an increase of millions of the data to be processed. In order to obtain valuable information, large-scale data can be processed and analyzed using a well-developed deep learning framework. This study introduces the concept of deep learning and three common deep learning models - Multilayer Perceptron, Convolutional Neural Network and Recurrent Neural Network, and analyzes the improvement of the model in dealing with large-scale data and gives the capacity and diversity analysis. Introducing the innovative application of deep learning in various fields under big data. Looking forward to the development of deep learning in the era of big data, the integration of big data and in-depth learning will make breakthroughs in various fields. Through constant innovation, they will gradually create more value for mankind.
Deep learning for high-resolution seismic imaging

Liyun Ma,Liguo Han,Qiang Feng

DOI: https://doi.org/10.1038/s41598-024-61251-8

IF: 4.6

2024-05-08

Scientific Reports

Abstract:Seismic imaging techniques play a crucial role in interpreting subsurface geological structures by analyzing the propagation and reflection of seismic waves. However, traditional methods face challenges in achieving high resolution due to theoretical constraints and computational costs. Leveraging recent advancements in deep learning, this study introduces a neural network framework that integrates Transformer and Convolutional Neural Network (CNN) architectures, enhanced through Adaptive Spatial Feature Fusion (ASFF), to achieve high-resolution seismic imaging. Our approach directly maps seismic data to reflection models, eliminating the need for post-processing low-resolution results. Through extensive numerical experiments, we demonstrate the outstanding ability of this method to accurately infer subsurface structures. Evaluation metrics including Root Mean Square Error (RMSE), Correlation Coefficient (CC), and Structural Similarity Index (SSIM) emphasize the model's capacity to faithfully reconstruct subsurface features. Furthermore, noise injection experiments showcase the reliability of this efficient seismic imaging method, further underscoring the potential of deep learning in seismic imaging.

multidisciplinary sciences
Data Driven Modeling and Prediction for Reservoir Characterization Using Seismic Attribute Analyses and Big Data Analytics

Xu Zhou,Mayank Tyagi,Guoyin Zhang,Hao Yu,Yangkang Chen

DOI: https://doi.org/10.2118/195856-ms

2019-01-01

Abstract:Abstract With recent developments in data acquisition and storage techniques, there exists huge amount of available data for data-driven decision making in the Oil & Gas industry. This study explores an application of using Big Data Analytics to establish the statistical relationships between seismic attribute values from a 3D seismic survey and petrophysical properties from well logs. Such relationships and models can be further used for the optimization of exploration and production operations. 3D seismic data can be used to extract various seismic attribute values at all locations within the seismic survey. Well logs provide accurate constraints on the petrophysical values along the wellbore. Big Data Analytics methods are utilized to establish the statistical relationships between seismic attributes and petrophysical data. Since seismic data are at the reservoir scale and are available at every sample cell of the seismic survey, these relationships can be used to estimate the petrophysical properties at all locations inside the seismic survey. In this study, the Teapot dome 3D seismic survey is selected to extract seismic attribute values. A set of instantaneous seismic attributes, including curvature, instantaneous phase, and trace envelope, are extracted from the 3D seismic volume. Deep Learning Neural Network models are created to establish the relationships between the input seismic attribute values from the seismic survey and petrophysical properties from well logs. Results show that a Deep Learning Neural Network model with multi-hidden layers is capable of predicting porosity values using extracted seismic attribute values from 3D seismic volumes. Ultilization of a subset of seismic attributes improves the model performance in predicting porosity values from seismic data.
Combining Deep Learning with Physics Based Features in Explosion-Earthquake Discrimination

Qingkai Kong,Ruijia Wang,William R. Walter,Moira Pyle,Keith Koper,Brandon Schmandt

DOI: https://doi.org/10.1029/2022GL098645

2022-03-12

Abstract:This paper combines the power of deep-learning with the generalizability of physics-based features, to present an advanced method for seismic discrimination between earthquakes and explosions. The proposed method contains two branches: a deep learning branch operating directly on seismic waveforms or spectrograms, and a second branch operating on physics-based parametric features. These features are high-frequency P/S amplitude ratios and the difference between local magnitude (ML) and coda duration magnitude (MC). The combination achieves better generalization performance when applied to new regions than models that are developed solely with deep learning. We also examined which parts of the waveform data dominate deep learning decisions (i.e., via Grad-CAM). Such visualization provides a window into the black-box nature of the machine-learning models and offers new insight into how the deep learning derived models use data to make the decisions.

Geophysics,Machine Learning
Foundation Models for Geophysics: Reviews and Perspectives

Qi Liu,Jianwei Ma

2024-01-01

Abstract:Recently, large models, or foundation models have demonstrated outstanding performance and have been applied in a variety of disciplines, such as chemistry, biology, economics, etc. Foundation models, trained on vast amounts of data, can be adapted to a wide range of use cases. The emergence of foundation models has a significant impact on the research paradigms in these fields. Geophysics is a scientific field dedicated to exploring and understanding the Earth's structures and states through the application of physical principles and the analysis of multimodal geophysical data. In the field of geophysics, the processing and interpretation of geophysical data are characterized by three primary features: extensive data volume, multimodality, and dependence on experience. These characteristics provide a suitable environment as well as challenges for the development and breakthrough of foundation models in the field of geophysics. In this perspective, we discuss the potential applications and research directions of geophysical foundation models (GeoFMs), exploring new research paradigms in geophysics in the era of foundation models. Exploration geophysics is the main focus, while the development of foundation models in remote sensing, seismology, and other related sub-disciplines in geophysics is also discussed. In the meantime, we also propose two strategies for constructing GeoFMs and discuss challenges that may arise during the process of development.

When Deep Learning Meets Geophysics

Deep Learning For Geophysics: Current And Future Trends

Finite Element Geotechnical Analysis Incorporating Deep Learning-Based Soil Model

Data-driven geophysics: from dictionary learning to deep learning

Deep Learning for Gravity and Magnetic Data Interpolation

Deep learning for geological hazards analysis: Data, models, applications, and opportunities

Improving AI System Awareness of Geoscience Knowledge: Symbiotic Integration of Physical Approaches and Deep Learning

Deep Learning for Spatiotemporal Big Data: A Vision on Opportunities and Challenges

Deep learning and its application in geochemical mapping

Deep-learning seismology

Deep learning and process understanding for data-driven Earth system science

Seismic Foundation Model (SFM): a new generation deep learning model in geophysics

Including Physics in Deep Learning -- An example from 4D seismic pressure saturation inversion

A 3D Geological Modeling Method Based on Geophysical Data

A Physics-guided Generative AI Toolkit for Geophysical Monitoring

Artificial intelligence for geoscience: Progress, challenges, and perspectives

Deep Learning Model And Its Application In Big Data

Deep learning for high-resolution seismic imaging

Data Driven Modeling and Prediction for Reservoir Characterization Using Seismic Attribute Analyses and Big Data Analytics

Combining Deep Learning with Physics Based Features in Explosion-Earthquake Discrimination

Foundation Models for Geophysics: Reviews and Perspectives