Abstract:With the emergence of the Smart City concept, the rapid advancement of urban three-dimensional (3D) reconstruction becomes imperative. While current developments in the field of 3D reconstruction have enabled the generation of 3D products such as Digital Surface Models (DSM), challenges persist in accurately reconstructing shadows, handling occlusions, and addressing low-texture areas in very-high-resolution remote sensing images. These challenges often lead to difficulties in calculating satisfactory disparity maps using existing stereo matching methods, thereby reducing the accuracy of 3D reconstruction. This issue is particularly pronounced in urban scenes, which contain numerous super high-rise and densely distributed buildings, resulting in large disparity values and occluded regions in stereo image pairs, and further leading to a large number of mismatched points in the obtained disparity map. In response to these challenges, this paper proposes a method to refine the disparity in urban scenes based on open-source GIS data. First, we register the GIS data with the epipolar-rectified images since there always exists unignorable geolocation errors between them. Specifically, buildings with different heights present different offsets in GIS data registering; thus, we perform multi-modal matching for each building and merge them into the final building mask. Subsequently, a two-layer optimization process is applied to the initial disparity map based on the building mask, encompassing both global and local optimization. Finally, we perform a post-correction on the building facades to obtain the final refined disparity map that can be employed for high-precision 3D reconstruction. Experimental results on SuperView-1, GaoFen-7, and GeoEye satellite images show that the proposed method has the ability to correct the occluded and mismatched areas in the initial disparity map generated by both hand-crafted and deep-learning stereo matching methods. The DSM generated by the refined disparity reduces the average height error from 2.2 m to 1.6 m, which demonstrates superior performance compared with other disparity refinement methods. Furthermore, the proposed method is able to improve the integrity of the target structure and present steeper building facades and complete roofs, which are conducive to subsequent 3D model generation.

A Multi-View Dense Image Matching Method for High-Resolution Aerial Imagery Based on A Graph Network

Multiresolution Image Matching Method Based on Graph-cut and Hausdorff Distance

Real-time and High Precision Feature Matching Between Blur Aerial Images

A Hypergraph Matching Framework for Refining Multi-source Feature Correspondences.

A Graph-Matching Approach for Cross-view Registration of Over-view 2 and Street-view based Point Clouds

Leveraging photogrammetric mesh models for aerial-ground feature point matching toward integrated 3D reconstruction

A Multi-View Dense Point Cloud Generation Algorithm Based on Low-Altitude Remote Sensing Images

Multi-view Feature Matching and Image Grouping from Multiple Unordered Wide-Baseline Images

Progressive matching method of aerial-ground remote sensing image via multi-scale context feature coding

A Dual Branch Multiscale Stereo Matching Network for High-Resolution Satellite Remote Sensing Images

Efficient match pair selection for matching large-scale oblique UAV images using spatial priors

UNSUPERVISED MULTI-CONSTRAINT DEEP NEURAL NETWORK FOR DENSE IMAGE MATCHING

Generalized Stereo Matching Method Based on Iterative Optimization of Hierarchical Graph Structure Consistency Cost for Urban 3D Reconstruction

Robust Fusion of Multi-Source Images for Accurate 3D Reconstruction of Complex Urban Scenes

Integrated Multi-View Stereo Matching System with Arbitrary Viewpoints

Disparity Refinement for Stereo Matching of High-Resolution Remote Sensing Images Based on GIS Data

Multi-View Stereo Via Geometric Expansion And Depth Refinement

Quasi-dense Matching Based on Multi-features in Continuous Scale for Approximate Epipolar Images

State of the art in dense image matching cost computation for high-resolution satellite stereo

High completeness multi-view stereo for dense reconstruction of large-scale urban scenes

A Hierarchical Image Matching Method For Stereo Satellite Imagery