Deep Learning Guided Building Reconstruction from Satellite Imagery-derived Point Clouds

Bo Xu,Xu Zhang,Zhixin Li,Matt Leotta,Shih-Fu Chang,Jie Shan

DOI: https://doi.org/10.48550/arXiv.2005.09223

2020-05-19

Abstract:3D urban reconstruction of buildings from remotely sensed imagery has drawn significant attention during the past two decades. While aerial imagery and LiDAR provide higher resolution, satellite imagery is cheaper and more efficient to acquire for large scale need. However, the high, orbital altitude of satellite observation brings intrinsic challenges, like unpredictable atmospheric effect, multi view angles, significant radiometric differences due to the necessary multiple views, diverse land covers and urban structures in a scene, small base-height ratio or narrow field of view, all of which may degrade 3D reconstruction quality. To address these major challenges, we present a reliable and effective approach for building model reconstruction from the point clouds generated from multi-view satellite images. We utilize multiple types of primitive shapes to fit the input point cloud. Specifically, a deep-learning approach is adopted to distinguish the shape of building roofs in complex and yet noisy scenes. For points that belong to the same roof shape, a multi-cue, hierarchical RANSAC approach is proposed for efficient and reliable segmenting and reconstructing the building point cloud. Experimental results over four selected urban areas (0.34 to 2.04 sq km in size) demonstrate the proposed method can generate detailed roof structures under noisy data environments. The average successful rate for building shape recognition is 83.0%, while the overall completeness and correctness are over 70% with reference to ground truth created from airborne lidar. As the first effort to address the public need of large scale city model generation, the development is deployed as open source software.

Computer Vision and Pattern Recognition,Machine Learning,Image and Video Processing

What problem does this paper attempt to address?

This paper attempts to solve the problem of reconstructing urban building models from point clouds generated from satellite images. Specifically, the paper aims to develop a robust method that can utilize point clouds generated from multi - view satellite images to efficiently and accurately reconstruct building models in large - scale scenarios (such as at the city scale). This involves dealing with the inherent challenges of satellite images, such as unpredictable atmospheric effects, multi - view angles, significant radiation differences, diverse surface coverages and urban structures in the scene, small base - height ratios or narrow fields of view, etc., all of which may reduce the quality of 3D reconstruction. To overcome these challenges, the paper proposes a reliable and effective building model reconstruction method. It uses multiple primitive shapes to fit the input point clouds and adopts a deep - learning method to distinguish building roof shapes in complex noisy scenes. For points belonging to the same roof shape, a multi - cue hierarchical RANSAC method is proposed to efficiently and reliably segment and reconstruct building point clouds.