Abstract:With the emergence of the Smart City concept, the rapid advancement of urban three-dimensional (3D) reconstruction becomes imperative. While current developments in the field of 3D reconstruction have enabled the generation of 3D products such as Digital Surface Models (DSM), challenges persist in accurately reconstructing shadows, handling occlusions, and addressing low-texture areas in very-high-resolution remote sensing images. These challenges often lead to difficulties in calculating satisfactory disparity maps using existing stereo matching methods, thereby reducing the accuracy of 3D reconstruction. This issue is particularly pronounced in urban scenes, which contain numerous super high-rise and densely distributed buildings, resulting in large disparity values and occluded regions in stereo image pairs, and further leading to a large number of mismatched points in the obtained disparity map. In response to these challenges, this paper proposes a method to refine the disparity in urban scenes based on open-source GIS data. First, we register the GIS data with the epipolar-rectified images since there always exists unignorable geolocation errors between them. Specifically, buildings with different heights present different offsets in GIS data registering; thus, we perform multi-modal matching for each building and merge them into the final building mask. Subsequently, a two-layer optimization process is applied to the initial disparity map based on the building mask, encompassing both global and local optimization. Finally, we perform a post-correction on the building facades to obtain the final refined disparity map that can be employed for high-precision 3D reconstruction. Experimental results on SuperView-1, GaoFen-7, and GeoEye satellite images show that the proposed method has the ability to correct the occluded and mismatched areas in the initial disparity map generated by both hand-crafted and deep-learning stereo matching methods. The DSM generated by the refined disparity reduces the average height error from 2.2 m to 1.6 m, which demonstrates superior performance compared with other disparity refinement methods. Furthermore, the proposed method is able to improve the integrity of the target structure and present steeper building facades and complete roofs, which are conducive to subsequent 3D model generation.

Real-Time 3D Map-Building Based on Stereo Vision

3-D Map Reconstruction in Cross-country Environment

3D Map Building for Mobile Robot under Unknown Outdoor Environments

Stereo Vision Based SLAM in Outdoor Environments

Stereo Visual Odometry from Disparity Space in Dynamic Environments

Real-time Stereo Vision System Using Adaptive Weight Cost Aggregation Approach

Stixel Based Urban Scene Representation by Stereo Vision

Accurate Real-Time Stereo Correspondence Using Intra- and Inter-Scanline Optimization

3d Stereoscopic Image Generation From Monocular View In Structured Environments

Realistic Terrain Visualization Based on 3D Virtual World Technology

A novel 2D-to-3D conversion method based on blocks world

Real-time Stereo to Multi-View Conversion System Based on Adaptive Meshing

Disparity Refinement for Stereo Matching of High-Resolution Remote Sensing Images Based on GIS Data

Binocular Stereo Vision Based 3D Mapping for Micro Aerial Vehicles in an Indoor Environment

Improving RealSense by Fusing Color Stereo Vision and Infrared Stereo Vision for the Visually Impaired

MRF-Based Stereo Correspondence and Virtual View Interpolation

Extracting Disparity Map from Bifocal Monocular Stereo Vision in a Novel Way

A Method of 3D-Gis Application Aided with Stereo Panorama Technology

Disparity Estimation Based on the Combination of Depth Camera and Stereo Vision

Depth Map Super-Resolution Using Stereo-Vision-Assisted Model

Adaptive Aggregation Based Cooperative Stereo Vision