Sensing a Priori Constraints in DNNs for Solving Geophysical Problems

Xinming Wu,Jianwei Ma,Xu Si,Zhengfa Bi,Jiarun Yang,Hui Gao,Dian-Lin Xie,Zhixin Guo,Jie Zhang

DOI: https://doi.org/10.5281/zenodo.7326606

2022-01-01

Abstract:Original data and associated results for the paper "Sensing a priori constraints in DNNs for solving geophysical problems" submitted to PNAS on Nov. 16th, 2022.

What problem does this paper attempt to address?

Sensing prior constraints in deep neural networks for solving exploration geophysical problems

Xinming Wu,Jianwei Ma,Xu Si,Zhengfa Bi,Jiarun Yang,Hui Gao,Dongzi Xie,Zhixiang Guo,Jie Zhang

DOI: https://doi.org/10.1073/pnas.2219573120

2023-06-02

Abstract:One of the key objectives in geophysics is to characterize the subsurface through the process of analyzing and interpreting geophysical field data that are typically acquired at the surface. Data-driven deep learning methods have enormous potential for accelerating and simplifying the process but also face many challenges, including poor generalizability, weak interpretability, and physical inconsistency. We present three strategies for imposing domain knowledge constraints on deep neural networks (DNNs) to help address these challenges. The first strategy is to integrate constraints into data by generating synthetic training datasets through geological and geophysical forward modeling and properly encoding prior knowledge as part of the input fed into the DNNs. The second strategy is to design nontrainable custom layers of physical operators and preconditioners in the DNN architecture to modify or shape feature maps calculated within the network to make them consistent with the prior knowledge. The final strategy is to implement prior geological information and geophysical laws as regularization terms in loss functions for training the DNNs. We discuss the implementation of these strategies in detail and demonstrate their effectiveness by applying them to geophysical data processing, imaging, interpretation, and subsurface model building.
Intelligent Reconstruction for Spatially Irregular Seismic Data by Combining Compressed Sensing with Deep Learning

Xinyue Gong,Shengchang Chen,Chengmei Jin

DOI: https://doi.org/10.3389/feart.2023.1299070

IF: 2.9

2023-01-01

Frontiers in Earth Science

Abstract:Data reconstruction is the most essential step in seismic data processing. Although the compressed sensing (CS) theory breaks through the Nyquist sampling theorem, we previously proved that the CS-based reconstruction of spatially irregular seismic data could not fully meet the theoretical requirements, resulting in low reconstruction accuracy. Although deep learning (DL) has great potential in mining features from data and accelerating the process, it faces challenges in earth science such as limited labels and poor generalizability. To improve the generalizability of deep neural network (DNN) in reconstructing seismic data in the actual situation of limited labeling, this paper proposes a method called CSDNN that combines model-driven CS and data-driven DNN to reconstruct the spatially irregular seismic data. By physically constraining neural networks, this method increases the generalizability of the network and improves the insufficient reconstruction caused by the inability to sample randomly in the whole data definition domain. Experiments on the synthetic and field seismic data show that the CSDNN reconstruction method achieves better performance compared with the conventional CS method and DNN method, including those with low sampling rates, which verifies the feasibility, effectiveness and generalizability of this approach.
Physics-Informed Neural Networks for Multiphysics Data Assimilation with Application to Subsurface Transport

QiZhi He,David Brajas-Solano,Guzel Tartakovsky,Alexandre M. Tartakovsky

DOI: https://doi.org/10.1016/j.advwatres.2020.103610

2019-12-06

Abstract:Data assimilation for parameter and state estimation in subsurface transport problems remains a significant challenge due to the sparsity of measurements, the heterogeneity of porous media, and the high computational cost of forward numerical models. We present a physics-informed deep neural networks (DNNs) machine learning method for estimating space-dependent hydraulic conductivity, hydraulic head, and concentration fields from sparse measurements. In this approach, we employ individual DNNs to approximate the unknown parameters (e.g., hydraulic conductivity) and states (e.g., hydraulic head and concentration) of a physical system, and jointly train these DNNs by minimizing the loss function that consists of the governing equations residuals in addition to the error with respect to measurement data. We apply this approach to assimilate conductivity, hydraulic head, and concentration measurements for joint inversion of the conductivity, hydraulic head, and concentration fields in a steady-state advection--dispersion problem. We study the accuracy of the physics-informed DNN approach with respect to data size, number of variables (conductivity and head versus conductivity, head, and concentration), DNNs size, and DNN initialization during training. We demonstrate that the physics-informed DNNs are significantly more accurate than standard data-driven DNNs when the training set consists of sparse data. We also show that the accuracy of parameter estimation increases as additional variables are inverted jointly.

Machine Learning,Computational Physics
Including Physics in Deep Learning -- An example from 4D seismic pressure saturation inversion

Jesper Sören Dramsch,Gustavo Corte,Hamed Amini,Colin MacBeth,Mikael Lüthje

DOI: https://doi.org/10.3997/2214-4609.201901967

2019-04-04

Abstract:Geoscience data often have to rely on strong priors in the face of uncertainty. Additionally, we often try to detect or model anomalous sparse data that can appear as an outlier in machine learning models. These are classic examples of imbalanced learning. Approaching these problems can benefit from including prior information from physics models or transforming data to a beneficial domain. We show an example of including physical information in the architecture of a neural network as prior information. We go on to present noise injection at training time to successfully transfer the network from synthetic data to field data.

Geophysics,Machine Learning,Data Analysis, Statistics and Probability
Physics-Informed neural network solver for numerical analysis in geoengineering

Xiao-Xuan Chen,Pin Zhang,Zhen-Yu Yin

DOI: https://doi.org/10.1080/17499518.2024.2315301

2024-02-21

Georisk Assessment and Management of Risk for Engineered Systems and Geohazards

Abstract:Engineering-scale problems generally can be described by partial differential equations (PDEs) or ordinary differential equations (ODEs). Analytical, semi-analytical and numerical analysis are commonly used for deriving the solutions of such PDEs/ODEs. Recently, a novel physics-informed neural network (PINN) solver has emerged as a promising alternative to solve PDEs/ODEs. PINN resembles a mesh-free method which leverages the strong non-linear ability of the deep learning algorithms (e.g. neural networks) to automatically search for the correct spatial-temporal responses constrained by embedded PDEs/ODEs. This study comprehensively reviews the current state of PINN including its principles for the forward and inverse problems, baseline algorithms for PINN, enhanced PINN variants combined with special sampling strategies and loss functions. PINN shows an easier modelling process and superior feasibility for inverse problems compared to conventional numerical methods. Meanwhile, the limitations and challenges of applications of current PINN solvers to constitutive modelling and multi-scale/phase problems are also discussed in terms of convergence ability and computational costs. PINN has exhibited its huge potential in geoengineering and brings a revolutionary way for numerous domain problems.

geosciences, multidisciplinary,engineering, geological
Neural Map Prior for Autonomous Driving

Xuan Xiong,Yicheng Liu,Tianyuan Yuan,Yue Wang,Yilun Wang,Hang Zhao

DOI: https://doi.org/10.48550/arXiv.2304.08481

2023-06-15

Abstract:High-definition (HD) semantic maps are crucial in enabling autonomous vehicles to navigate urban environments. The traditional method of creating offline HD maps involves labor-intensive manual annotation processes, which are not only costly but also insufficient for timely updates. Recent studies have proposed an alternative approach that generates local maps using online sensor observations. However, this approach is limited by the sensor's perception range and its susceptibility to occlusions. In this study, we propose Neural Map Prior (NMP), a neural representation of global maps. This representation automatically updates itself and improves the performance of local map inference. Specifically, we utilize two approaches to achieve this. Firstly, to integrate a strong map prior into local map inference, we apply cross-attention, a mechanism that dynamically identifies correlations between current and prior features. Secondly, to update the global neural map prior, we utilize a learning-based fusion module that guides the network in fusing features from previous traversals. Our experimental results, based on the nuScenes dataset, demonstrate that our framework is highly compatible with various map segmentation and detection architectures. It significantly improves map prediction performance, even in challenging weather conditions and situations with a longer perception range. To the best of our knowledge, this is the first learning-based system for creating a global map prior.

Computer Vision and Pattern Recognition
Robust deep learning seismic inversion with a priori initial model constraint

Jian Zhang,Jingye Li,Xiaohong Chen,Yuanqiang Li,Guangtan Huang,Yangkang Chen

DOI: https://doi.org/10.1093/gji/ggab074

IF: 3.352

2021-02-20

Geophysical Journal International

Abstract:SUMMARY Seismic inversion is one of the most commonly used methods in the oil and gas industry for reservoir characterization from observed seismic data. Deep learning (DL) is emerging as a data-driven approach that can effectively solve the inverse problem. However, existing DL-based methods for seismic inversion utilize only seismic data as input, which often leads to poor stability of the inversion results. Besides, it has always been challenging to train a robust network since the real survey has limited labelled data pairs. To partially overcome these issues, we develop a neural network framework with a priori initial model constraint to perform seismic inversion. Our network uses two parts as one input for training. One is the seismic data, and the other is the subsurface background model. The labels for each input are the actual model. The proposed method is performed by log-to-log strategy. The training data set is first generated based on forward modelling. The network is then pre-trained using the synthetic training data set, which is further validated using synthetic data that have not been used in the training step. After obtaining the pre-trained network, we introduce the transfer learning strategy to fine-tune the pre-trained network using labelled data pairs from a real survey to acquire better inversion results in the real survey. The validity of the proposed framework is demonstrated using synthetic 2-D data including both post-stack and pre-stack examples, as well as a real 3-D post-stack seismic data set from the western Canadian sedimentary basin.

geochemistry & geophysics
Transparent Object Sensing With Enhanced Prior From Deep Convolutional Neural Network

Jing Wang,Jian Bai,Xiao Huang,Xiangdong Zhou,Lei Zhao,Kun Yan,Jing Hou,Kailun Yang

DOI: https://doi.org/10.1117/12.2533173

2019-01-01

Abstract:In recent years, with the development of new materials, transparent objects are playing an increasingly important role in many fields, from industrial manufacturing to military technology. However, transparent objects sensing still remains a challenging problem in the area of computational imaging and optical engineering. As an indispensable part of 3-D modeling, transparent object sensing is a long-standing research topic, which aims to reconstruct the surface shape of a given transparent object using various kinds of measurement methods. In this paper, we put forward a new method for the sensing of such objects. Specifically, we focus on the sensing of thin transparent objects, including thin films and various kinds of nano-materials. The proposed method consists of two main steps. Firstly, we use a deep convolutional neural network to predict the original distribution of the objects from its recorded intensity pattern. Secondly, the predicted results are used as initial estimates, and the iterative projection phase retrieval algorithm is performed with the enhanced priors to obtain finer reconstruction results. The numerical experiment results turned out that, with the two steps, our method is able to reconstruct the surface shape of a given thin transparent object with a high speed and simple experimental setup. Moreover, the proposed method shows a new path of transparent object sensing with the combination of state-of-art deep learning technique and conventional computational imaging algorithm. It indicates that, following the same framework, the performance of such method can be significantly improved with more advanced hardware and software implementation.
Solving Partial Differential Equations with Point Source Based on Physics-Informed Neural Networks

Xiang Huang,Hongsheng Liu,Beiji Shi,Zidong Wang,Kang Yang,Yang Li,Bingya Weng,Min Wang,Haotian Chu,Jing Zhou,Fan Yu,Bei Hua,Lei Chen,Bin Dong

2021-11-02

Abstract:In recent years, deep learning technology has been used to solve partial differential equations (PDEs), among which the physics-informed neural networks (PINNs) emerges to be a promising method for solving both forward and inverse PDE problems. PDEs with a point source that is expressed as a Dirac delta function in the governing equations are mathematical models of many physical processes. However, they cannot be solved directly by conventional PINNs method due to the singularity brought by the Dirac delta function. We propose a universal solution to tackle this problem with three novel techniques. Firstly the Dirac delta function is modeled as a continuous probability density function to eliminate the singularity; secondly a lower bound constrained uncertainty weighting algorithm is proposed to balance the PINNs losses between point source area and other areas; and thirdly a multi-scale deep neural network with periodic activation function is used to improve the accuracy and convergence speed of the PINNs method. We evaluate the proposed method with three representative PDEs, and the experimental results show that our method outperforms existing deep learning-based methods with respect to the accuracy, the efficiency and the versatility.

Artificial Intelligence,Computational Physics
A multi-view ensemble machine learning approach for 3D modeling using geological and geophysical data

Deping Chu Jinming Fu Bo Wan Hong Li Lulan Li Fang Fang Shengwen Li Shengyong Pan Shunping Zhou a Faculty of Geography and Information Engineering,China University of Geosciences,Wuhan,Chinab School of Computer Science,China University of Geosciences,Wuhan,Chinac National Engineering Research Center of Geographic Information System,Wuhan,Chinad Wuhan Zondy Cyber,Wuhan,ChinaDeping Chu is a PhD at China University of Geosciences,Wuhan. His research interests include geoscience big data mining,geoscience knowledge graph,and geographic information science. The main contributions to the paper are conceptualization,methodology,writing-original draft.Jinming Fu is a master at China University of Geosciences,Wuhan. His research interests include geospatial modeling techniques and machine learning. The main contributions to the paper are investigation and software.Bo Wan is a professor at the School of Computer Science,China University of Geosciences,Wuhan. His research interests include geographic information science,geospatial modeling techniques,and geoscience process simulation. The main contributions to the paper are writing-review,editing,and supervision.Hong Li is a PhD at China University of Geosciences,Wuhan. Her research interests include geospatial modeling techniques and geoscience process simulation. The main contributions to the paper are writing-review and supervision.Lulan Li is a master at China University of Geosciences,Wuhan. Her research interests include machine learning and geospatial modeling techniques. The main contribution to the paper is supervision.Fang Fang is an associate professor at School of Computer Science,China University of Geosciences. Her research interests include intelligent interpretation of remote sensing imagery and urban visual intelligence. The main contribution to the paper is supervision.Shengwen Li is an associate professor at School of Computer Science,China University of Geosciences. His research interests include deep learning for remote sensing,spatial-temporal data mining,and knowledge graph. The main contribution to the paper is supervision.Shengyong Pan graduated from China University of Geosciences in 2006 with a master's degree in Cartography and Geographic Information Engineering,and is currently working at Wuhan Zondy Cyber. His research interests focus on geological informatization and 3D modeling. The main contribution to the paper is supervision.Shunping Zhou is a professor at the School of Computer Science,China University of Geosciences,Wuhan. His research interests include spatial database technology,geospatial artificial intelligence,and computer vision. The main contribution to the paper is supervision.

DOI: https://doi.org/10.1080/13658816.2024.2394228

2024-08-25

International Journal of Geographical Information Science

Abstract:Geophysical data are often integrated into geological data for 3D modeling of underground spaces. However, the existing single-view approach means it is difficult to adequately fuse the valid information between the two types of data, and the complexity of lithological decoding and classification is high. To address this issue, a multi-view ensemble machine learning (ML) framework is proposed. Initially, the original dataset of lithology prediction is constructed by aligning geological and geophysical data with different spatial scales. Next, the dataset is divided into three datasets of structural strength, density, and moisture content according to the lithology properties of the geophysical data. The proposed framework is then used to capture the lithologic characteristics under different views to achieve the prediction of lithologic labels. In this process, a self-attentive mechanism is used to adaptively fuse the valid information under each view. To validate the proposed framework, it is applied to a project in Jiaxing, Zhejiang Province, China. Compared with existing ML methods, the proposed multi-view ensemble ML framework improves modeling accuracy and constructs models with low uncertainty. The framework can be extended to other multi-source data fusion tasks across geoscience domains.

geography, physical,computer science, information systems,information science & library science
Joint Gravity and Magnetic Inversion Using CNNs' Deep Learning

Zhijing Bai,Yanfei Wang,Chenzhang Wang,Caixia Yu,Dmitry Lukyanenko,Inna Stepanova,Anatoly G. Yagola

DOI: https://doi.org/10.3390/rs16071115

IF: 5

2024-03-23

Remote Sensing

Abstract:Enhancing the reliability of inversion results has always been a prominent issue in the field of geophysics. In recent years, data-driven inversion methods leveraging deep neural networks (DNNs) have gained prominence for their ability to address non-uniqueness issues and reduce computational costs compared to traditional physically model-driven methods. In this study, we propose a GMNet machine learning method, i.e., a CNN-based inversion method for gravity and magnetic field data. This method relies more on data-driven training, and in the prediction phase after the model is trained, it does not heavily depend on a priori assumptions, unlike traditional methods. By forward modeling gravity and magnetic fields, we obtain a substantial dataset to train the CNN model, enabling the direct mapping from field data to subsurface property distribution. Applying this method to synthetic data and one-field data yields promising inversion results.

environmental sciences,imaging science & photographic technology,remote sensing,geosciences, multidisciplinary
Hyperspectral Denoising Using Unsupervised Disentangled Spatio-Spectral Deep Priors

Yu-Chun Miao,Xi-Le Zhao,Xiao Fu,Jian-Li Wang,Yu-Bang Zheng

DOI: https://doi.org/10.1109/TGRS.2021.3106380

2021-08-19

Abstract:Image denoising is often empowered by accurate prior information. In recent years, data-driven neural network priors have shown promising performance for RGB natural image denoising. Compared to classic handcrafted priors (e.g., sparsity and total variation), the "deep priors" are learned using a large number of training samples -- which can accurately model the complex image generating process. However, data-driven priors are hard to acquire for hyperspectral images (HSIs) due to the lack of training data. A remedy is to use the so-called unsupervised deep image prior (DIP). Under the unsupervised DIP framework, it is hypothesized and empirically demonstrated that proper neural network structures are reasonable priors of certain types of images, and the network weights can be learned without training data. Nonetheless, the most effective unsupervised DIP structures were proposed for natural images instead of HSIs. The performance of unsupervised DIP-based HSI denoising is limited by a couple of serious challenges, namely, network structure design and network complexity. This work puts forth an unsupervised DIP framework that is based on the classic spatio-spectral decomposition of HSIs. Utilizing the so-called linear mixture model of HSIs, two types of unsupervised DIPs, i.e., U-Net-like network and fully-connected networks, are employed to model the abundance maps and endmembers contained in the HSIs, respectively. This way, empirically validated unsupervised DIP structures for natural images can be easily incorporated for HSI denoising. Besides, the decomposition also substantially reduces network complexity. An efficient alternating optimization algorithm is proposed to handle the formulated denoising problem. Semi-real and real data experiments are employed to showcase the effectiveness of the proposed approach.

Image and Video Processing,Computer Vision and Pattern Recognition
Neural SPDE solver for uncertainty quantification in high-dimensional space-time dynamics

Maxime Beauchamp,Ronan Fablet,Hugo Georgenthum

2023-11-03

Abstract:Historically, the interpolation of large geophysical datasets has been tackled using methods like Optimal Interpolation (OI) or model-based data assimilation schemes. However, the recent connection between Stochastic Partial Differential Equations (SPDE) and Gaussian Markov Random Fields (GMRF) introduced a novel approach to handle large datasets making use of sparse precision matrices in OI. Recent advancements in deep learning also addressed this issue by incorporating data assimilation into neural architectures: it treats the reconstruction task as a joint learning problem involving both prior model and solver as neural networks. Though, it requires further developments to quantify the associated uncertainties. In our work, we leverage SPDEbased Gaussian Processes to estimate complex prior models capable of handling nonstationary covariances in space and time. We develop a specific architecture able to learn both state and SPDE parameters as a neural SPDE solver, while providing the precisionbased analytical form of the SPDE sampling. The latter is used as a surrogate model along the data assimilation window. Because the prior is stochastic, we can easily draw samples from it and condition the members by our neural solver, allowing flexible estimation of the posterior distribution based on large ensemble. We demonstrate this framework on realistic Sea Surface Height datasets. Our solution improves the OI baseline, aligns with neural prior while enabling uncertainty quantification and online parameter estimation.

Image and Video Processing
In-Place Panoptic Radiance Field Segmentation with Perceptual Prior for 3D Scene Understanding

Shenghao Li

2024-10-06

Abstract:Accurate 3D scene representation and panoptic understanding are essential for applications such as virtual reality, robotics, and autonomous driving. However, challenges persist with existing methods, including precise 2D-to-3D mapping, handling complex scene characteristics like boundary ambiguity and varying scales, and mitigating noise in panoptic pseudo-labels. This paper introduces a novel perceptual-prior-guided 3D scene representation and panoptic understanding method, which reformulates panoptic understanding within neural radiance fields as a linear assignment problem involving 2D semantics and instance recognition. Perceptual information from pre-trained 2D panoptic segmentation models is incorporated as prior guidance, thereby synchronizing the learning processes of appearance, geometry, and panoptic understanding within neural radiance fields. An implicit scene representation and understanding model is developed to enhance generalization across indoor and outdoor scenes by extending the scale-encoded cascaded grids within a reparameterized domain distillation framework. This model effectively manages complex scene attributes and generates 3D-consistent scene representations and panoptic understanding outcomes for various scenes. Experiments and ablation studies under challenging conditions, including synthetic and real-world scenes, demonstrate the proposed method's effectiveness in enhancing 3D scene representation and panoptic segmentation accuracy.

Computer Vision and Pattern Recognition
Digging into Depth Priors for Outdoor Neural Radiance Fields

Chen Wang,Jiadai Sun,Lina Liu,Chenming Wu,Zhelun Shen,Dayan Wu,Yuchao Dai,Liangjun Zhang

2023-08-09

Abstract:Neural Radiance Fields (NeRF) have demonstrated impressive performance in vision and graphics tasks, such as novel view synthesis and immersive reality. However, the shape-radiance ambiguity of radiance fields remains a challenge, especially in the sparse viewpoints setting. Recent work resorts to integrating depth priors into outdoor NeRF training to alleviate the issue. However, the criteria for selecting depth priors and the relative merits of different priors have not been thoroughly investigated. Moreover, the relative merits of selecting different approaches to use the depth priors is also an unexplored problem. In this paper, we provide a comprehensive study and evaluation of employing depth priors to outdoor neural radiance fields, covering common depth sensing technologies and most application ways. Specifically, we conduct extensive experiments with two representative NeRF methods equipped with four commonly-used depth priors and different depth usages on two widely used outdoor datasets. Our experimental results reveal several interesting findings that can potentially benefit practitioners and researchers in training their NeRF models with depth priors. Project Page: <a class="link-external link-https" href="https://cwchenwang.github.io/outdoor-nerf-depth" rel="external noopener nofollow">this https URL</a>

Computer Vision and Pattern Recognition
Reconstructing the Unsaturated Flow Equation from Sparse and Noisy Data: Leveraging the Synergy of Group Sparsity and Physics‐Informed Deep Learning

Wenxiang Song,Liangsheng Shi,Xiaolong Hu,Yanling Wang,Lijun Wang

DOI: https://doi.org/10.1029/2022WR034122

IF: 5.4

2023-05-05

Water Resources Research

Abstract:Data‐driven scientific discovery methods have been developed and applied to discover governing equations from data, involving the attempt to discover the unsaturated flow equation in soils from data. However, an important but unresolved problem is how to reconstruct the unsaturated flow equation from highly noisy and scarce discrete data. In this study, we present a new deep‐learning framework: DeepGS (Deep‐learning‐based Group Sparsity framework), that leverages the synergy of group sparsity and physics‐informed deep learning to the latent governing equation for unsaturated flow. In particular, we design a strategy that decomposes the identification of the unsaturated flow equation into two tasks: the determination of the PDE structure and the reconstruction of the nonlinear coefficients. The tasks can be seamlessly handled by group sparse regression and the physics‐informed deep learning approach. Through the training, it realizes the simultaneous reconstruction of soil moisture dynamics and unsaturated flow governing equation. A series of comprehensive numerical experiments are conducted to determine the optimal architecture and test its performance. The results show the efficacy and robustness of DeepGS, which significantly outperform previous methods. We also conclude that accurately reconstructing soil moisture dynamics and spatiotemporal derivatives from noisy and scarce data play a critical role in governing equation discovery. This study further demonstrates the potential of discovering the governing equation for unsaturated flow from data in more complex scenarios, where rich and accurate soil moisture observations are generally intractable to access.

environmental sciences,water resources,limnology
3-D Gravity Inversion Based on Cross Dimensional Deep Neural Networks

Lianzhi Zhang,Guibin Zhang

DOI: https://doi.org/10.1190/iwmg2021-43.1

2022-01-01

Abstract:Three-dimensional (3D) gravity inversion obtains the density distribution of subsurface source through observed gravity field. The existing physics-driven methods are limited by the initial model and high computational cost. Recently, data-driven methods based on deep neural networks have received considerable attentions for geophysical inverse problems and reached a good performance both at speed and accuracy. Most of the deep-learning inversion strategies are applicable for two-dimensional (2D) subsurface imaging. But 3D gravity inversion is a cross dimensional problem. In this study, a novel 3D gravity inversion method based on encoder-decoder neural networks is proposed. The network has a gravity field encoder for 2D gravity field feature extraction, a dimension transformation module for feature space transformation, and a density structure decoder for 3D density structure reconstruction. Furthermore, a cross dimensional skip connection structure is proposed to improve the network. The improved network is called cross dimensional U-Net (CDU-Net). A highly random dataset is constructed to enlarge the feature distribution space of the samples and noise is added to make the data closer to the real scenarios. Numerical examples using synthetic gravity data show that the accuracy of the network can reach 96.6% in the entire area and 65.9% in the area of density structures while reducing the computational time by 20 times. In the experiments on real data of Vinton salt dome, the results are in good agreement with the known geological information.
Deep-Learning Inversion of Seismic Data

Shucai Li,Bin Liu,Yuxiao Ren,Yangkang Chen,Senlin Yang,Yunhai Wang,Peng Jiang

DOI: https://doi.org/10.1109/TGRS.2019.2953473

2020-06-26

Abstract:We propose a new method to tackle the mapping challenge from time-series data to spatial image in the field of seismic exploration, i.e., reconstructing the velocity model directly from seismic data by deep neural networks (DNNs). The conventional way of addressing this ill-posed inversion problem is through iterative algorithms, which suffer from poor nonlinear mapping and strong nonuniqueness. Other attempts may either import human intervention errors or underuse seismic data. The challenge for DNNs mainly lies in the weak spatial correspondence, the uncertain reflection-reception relationship between seismic data and velocity model, as well as the time-varying property of seismic data. To tackle these challenges, we propose end-to-end seismic inversion networks (SeisInvNets) with novel components to make the best use of all seismic data. Specifically, we start with every seismic trace and enhance it with its neighborhood information, its observation setup, and the global context of its corresponding seismic profile. From the enhanced seismic traces, the spatially aligned feature maps can be learned and further concatenated to reconstruct a velocity model. In general, we let every seismic trace contribute to the reconstruction of the whole velocity model by finding spatial correspondence. The proposed SeisInvNet consistently produces improvements over the baselines and achieves promising performance on our synthesized and proposed SeisInv data set according to various evaluation metrics. The inversion results are more consistent with the target from the aspects of velocity values, subsurface structures, and geological interfaces. Moreover, the mechanism and the generalization of the proposed method are discussed and verified. Nevertheless, the generalization of deep-learning-based inversion methods on real data is still challenging and considering physics may be one potential solution.

Computer Vision and Pattern Recognition,Artificial Intelligence
An E-PINN assisted practical uncertainty quantification for inverse problems

Xinchao Jiang,Xin Wanga,Ziming Wena,Enying Li,Hu Wang

DOI: https://doi.org/10.48550/arXiv.2209.10195

2022-09-21

Numerical Analysis

Abstract:How to solve inverse problems is the challenge of many engineering and industrial applications. Recently, physics-informed neural networks (PINNs) have emerged as a powerful approach to solve inverse problems efficiently. However, it is difficult for PINNs to quantify the uncertainty of results. Therefore, this study proposed ensemble PINNs (E-PINNs) to handle this issue. The E-PINN uses ensemble statistics of several basic models to provide uncertainty quantifications for the inverse solution based on the PINN framework, and it is employed to solve the inverse problems in which the unknown quantity is propagated through partial differential equations (PDEs), especially the identification of the unknown field (e.g., space function) of a given physical system. Compared with other data-driven approaches, the suggested method is more than straightforward to implement, and also obtains high-quality uncertainty estimates of the quantity of interest (QoI) without significantly increasing the complexity of the algorithm. This work discusses the good properties of ensemble learning in field inversion and uncertainty quantification. The effectiveness of the proposed method is demonstrated through several numerical experiments. To enhance the robustness of models, adversarial training (AT) is applied. Furthermore, an adaptive active sampling (AS) strategy based on the uncertainty estimates from E-PINNs is also proposed to improve the accuracy of material field inversion problems.
Hyperspectral Denoising Using Unsupervised Disentangled Spatiospectral Deep Priors

Yu-Chun Miao,Xi-Le Zhao,Xiao Fu,Jian-Li Wang,Yu-Bang Zheng

DOI: https://doi.org/10.1109/tgrs.2021.3106380

IF: 8.2

2022-01-01

IEEE Transactions on Geoscience and Remote Sensing

Abstract:Image denoising is often empowered by accurate prior information. In recent years, data-driven neural network priors have shown promising performance for RGB natural image denoising. Compared to classic handcrafted priors (e.g., sparsity and total variation), the "deep priors" are learned using a large number of training samples, which can accurately model the complex image generating process. However, data-driven priors are hard to acquire for hyperspectral images (HSIs) due to the lack of training data. A remedy is to use the so-called unsupervised deep image prior (DIP). Under the unsupervised DIP framework, it is hypothesized and empirically demonstrated that proper neural network structures are reasonable priors of certain types of images, and the network weights can be learned without training data. Nonetheless, the most effective unsupervised DIP structures were proposed for natural images instead of HSIs. The performance of unsupervised DIP-based HSI denoising is limited by a couple of serious challenges, namely network structure design and network complexity. This work puts forth an unsupervised DIP framework that is based on the classic spatiospectral decomposition of HSIs. Utilizing the so-called linear mixture model of HSIs, two types of unsupervised DIPs, that is, U-Net-like network and fully connected networks, are employed to model the abundance maps and endmembers contained in the HSIs, respectively. This way, empirically validated unsupervised DIP structures for natural images can be easily incorporated for HSI denoising. Besides, the decomposition also substantially reduces network complexity. An efficient alternating optimization algorithm is proposed to handle the formulated denoising problem. Simulated and real data experiments are employed to showcase the effectiveness of the proposed approach.

imaging science & photographic technology,remote sensing,engineering, electrical & electronic,geochemistry & geophysics

Sensing a Priori Constraints in DNNs for Solving Geophysical Problems

Sensing prior constraints in deep neural networks for solving exploration geophysical problems

Intelligent Reconstruction for Spatially Irregular Seismic Data by Combining Compressed Sensing with Deep Learning

Physics-Informed Neural Networks for Multiphysics Data Assimilation with Application to Subsurface Transport

Including Physics in Deep Learning -- An example from 4D seismic pressure saturation inversion

Physics-Informed neural network solver for numerical analysis in geoengineering

Neural Map Prior for Autonomous Driving

Robust deep learning seismic inversion with a priori initial model constraint

Transparent Object Sensing With Enhanced Prior From Deep Convolutional Neural Network

Solving Partial Differential Equations with Point Source Based on Physics-Informed Neural Networks

A multi-view ensemble machine learning approach for 3D modeling using geological and geophysical data

Joint Gravity and Magnetic Inversion Using CNNs' Deep Learning

Hyperspectral Denoising Using Unsupervised Disentangled Spatio-Spectral Deep Priors

Neural SPDE solver for uncertainty quantification in high-dimensional space-time dynamics

In-Place Panoptic Radiance Field Segmentation with Perceptual Prior for 3D Scene Understanding

Digging into Depth Priors for Outdoor Neural Radiance Fields

Reconstructing the Unsaturated Flow Equation from Sparse and Noisy Data: Leveraging the Synergy of Group Sparsity and Physics‐Informed Deep Learning

3-D Gravity Inversion Based on Cross Dimensional Deep Neural Networks

Deep-Learning Inversion of Seismic Data

An E-PINN assisted practical uncertainty quantification for inverse problems

Hyperspectral Denoising Using Unsupervised Disentangled Spatiospectral Deep Priors