Abstract:In this paper, we propose a general deep learning based framework, named Sat-MVSF, to perform three-dimensional (3D) reconstruction of the Earth's surface from multi-view optical satellite images. The framework is a complete processing pipeline, including pre-processing, a multi-view stereo (MVS) network for satellite imagery (Sat-MVSNet), and post-processing. The pre-processing handles the geometric and radiometric configuration of the multi-view images and their cropping. The cropped multi-view patches are then fed into Sat-MVSNet, which includes deep feature extraction, rational polynomial camera (RPC) warping, pyramid cost volume construction, regularization, and regression, to obtain the height maps. The error matches are then filtered out and a digital surface model (DSM) is generated in the post-processing. Considering the complexity and diversity of real-world scenes, we also introduce a self-refinement strategy that does not require any ground-truth labels to enhance the performance and robustness of the Sat-MVSF framework. We comprehensively compare the proposed framework with popular commercial software and open-source methods, to demonstrate the potential of the proposed deep learning framework. On the WHU-TLC dataset, where the images are captured with a three-line camera (TLC), the proposed framework outperforms all the other solutions in terms of reconstruction fineness, and also outperforms most of the other methods in terms of efficiency. On the challenging MVS3D dataset, where the images are captured by the WorldView-3 satellite at different times and seasons, the proposed framework also exceeds the existing methods when using the model pretrained on aerial images and the introduced self-refinement strategy, demonstrating a high generalization ability. We also note that the lack of training samples hinders research in this field, and the availability of more high-quality open-source training data will greatly accelerate the research into deep learning based MVS satellite image reconstruction. The code will be available at https://gpcv.whu.edu.cn/data.

Sub3DNet1.0: a deep-learning model for regional-scale 3D subsurface structure mapping

DeepISMNet: three-dimensional implicit structural modeling with convolutional neural network

Data-driven construction of Three-dimensional subsurface geological models from limited Site-specific boreholes and prior geological knowledge for underground digital twin

Deep Subsurface Pseudo-Lithostratigraphic Modeling Based on Three-Dimensional Convolutional Neural Network (3D CNN) Using Inversed Geophysical Properties and Shallow Subsurface Geological Model

Near real-time subsurface structure imaging using airborne electromagnetic data based on deep learning

Research on 3D Geological Modeling Method Based on Deep Neural Networks for Drilling Data

Generating subsurface earth models using discrete representation learning and deep autoregressive network

A Three-Dimensional Geological Structure Modeling Framework and Its Application in Machine Learning

Building subsurface models with horizon-guided interpolation and deep learning: Application to the Volve field

Subsurface Depths Structure Maps Reconstruction with Generative Adversarial Networks

3D Reconstruction of Remote Sensing Mountain Areas with TSDF-Based Neural Networks

Deep learning for high-resolution seismic imaging

Deep Neural Network for 3D Surface Segmentation based on Contour Tree Hierarchy

Deep learning-based geological map generation using geological routes

Remote sensing framework for geological mapping via stacked autoencoders and clustering

Reconstructing High-Resolution DEMs from 3D Terrain Features Using Conditional Generative Adversarial Networks

DEM super-resolution framework based on deep learning: decomposing terrain trends and residuals

Three-Dimensional Magnetotelluric Forward Modeling Through Deep Learning

Highly Applicable Iterative Network for 3-D Reconstruction Based on Multiview Satellite Images

A general deep learning based framework for 3D reconstruction from multi-view stereo satellite images

A Novel Recurrent Encoder-Decoder Structure for Large-Scale Multi-View Stereo Reconstruction From an Open Aerial Dataset