Abstract:Applying deep learning methods on high‐resolution satellite stereos for large‐scale and high‐fidelity DSM generation is still challenging. This paper proposes a novel HF2Net for image stereo matching and establishes a deep learning‐based DSM generation workflow. Experiments on stereo matching datasets and two Gaofen‐7 stereos demonstrate the proposed workflow's effectiveness and practical capability. Recent studies have demonstrated that deep learning‐based stereo matching methods (DLSMs) can far exceed conventional ones on most benchmark datasets by both improving visual performance and decreasing the mismatching rate. However, applying DLSMs on high‐resolution satellite stereos with broad image coverage and wide terrain variety is still challenging. First, the broad coverage of satellite stereos brings a wide disparity range, while DLSMs are limited to a narrow disparity range in most cases, resulting in incorrect disparity estimation in areas with contradictory disparity ranges. Second, high‐resolution satellite stereos always comprise various terrain types, which is more complicated than carefully prepared datasets. Thus, the performance of DLSMs on satellite stereos is unstable, especially for intractable regions such as texture‐less and occluded regions. Third, generating DSMs requires occlusion‐aware disparity maps, while traditional occlusion detection methods are not always applicable for DLSMs with continuous disparity. To tackle these problems, this paper proposes a novel DLSM‐based DSM generation workflow. The workflow comprises three steps: pre‐processing, disparity estimation and post‐processing. The pre‐processing step introduces low‐resolution terrain to shift unmatched disparity ranges into a fixed scope and crops satellite stereos to regular patches. The disparity estimation step proposes a hybrid feature fusion network (HF2Net) to improve the matching performance. In detail, HF2Net designs a cross‐scale feature extractor (CSF) and a multi‐scale cost filter. The feature extractor differentiates structural‐context features in complex scenes and thus enhances HF2Net's robustness to satellite stereos, especially on intractable regions. The cost filter filters out most matching errors to ensure accurate disparity estimation. The post‐processing step generates initial DSM patches with estimated disparity maps and then refines them for the final large‐scale DSMs. Primary experiments on the public US3D dataset showed better accuracy than state‐of‐the‐art methods, indicating HF2Net's superiority. We then created a self‐made Gaofen‐7 dataset to train HF2Net and conducted DSM generation experiments on two Gaofen‐7 stereos to further demonstrate the effectiveness and practical capability of the proposed workflow.

A General Deep Learning Framework Guided by Sparse Matching for Disparity Estimation in High-Resolution Satellite Stereo Imagery

Disparity Estimation Using Multilevel and Global Information

A Depth Estimation Framework Based on Unsupervised Learning and Cross-Modal Translation

End-to-End Edge-Guided Multi-Scale Matching Network for Optical Satellite Stereo Image Pairs

Stereo Matching Method with Integrated Geometric Encoding for Disparity Refinement

Expanding Sparse LiDAR Depth and Guiding Stereo Matching for Robust Dense Depth Estimation

Superpixel Guided Network for Three-Dimensional Stereo Matching

Deep Learning Meets Satellite Images -- An Evaluation on Handcrafted and Learning-based Features for Multi-date Satellite Stereo Images

SR-Stereo & DAPE: Stepwise Regression and Pre-trained Edges for Practical Stereo Matching

FINE-TUNING DEEP LEARNING MODELS FOR STEREO MATCHING USING RESULTS FROM SEMI-GLOBAL MATCHING

DEEP LEARNING-BASED STEREO MATCHING FOR HIGH-RESOLUTION SATELLITE IMAGES: A COMPARATIVE EVALUATION

Digital surface model generation from high‐resolution satellite stereos based on hybrid feature fusion network

A unified and efficient semi-supervised learning framework for stereo matching

SegStereo: Exploiting Semantic Information for Disparity Estimation

Disparity Refinement for Stereo Matching of High-Resolution Remote Sensing Images Based on GIS Data

Self-Supervised Multiscale Adversarial Regression Network for Stereo Disparity Estimation

Depth Edge and Structure Optimization-Based End-to-End Self-Supervised Stereo Matching

A Novel Cell Structure‐based Disparity Estimation for Unsupervised Stereo Matching

Towards accurate binocular vision of satellites: A Cascaded Multi-Scale Pyramid Network for stereo matching on satellite imagery

Stereo-LiDAR Depth Estimation with Deformable Propagation and Learned Disparity-Depth Conversion

A Learning-based Framework for Hybrid Depth-from-Defocus and Stereo Matching