Abstract:3-D building models play a vital role in numerous applications including urban planning and smart cities. Recent 3-D building modeling methods either rely heavily on available manually collected footprint references or hardly reach real automation on par with manual editing. To approach the automated extraction of instance-level 3-D buildings at level of detail 1 (LoD1), we introduce an innovative end-to-end 3-D building instance segmentation model. This model predicts accurate contours and heights of individual buildings simultaneously using ortho-rectified high-resolution remote sensing images and digital surface models (DSMs), getting rid of additional reference data and empirical parameter settings. First, we propose an anchor-free multihead (AFM) building extraction network tailored for extracting 2-D building contours. AFM incorporates a full-resolution, long-range correlation boosted global mask prediction branch along with anchor-free bounding box generation, as well as a newly developed online hard sample mining (OHSM) training procedure based on uncertainty analysis to emphasize error-prone positions in locating building contours. Subsequently, we incorporate a height prediction component to AFM in order to derive accurate building height information, thus creating the comprehensive 3-D building extraction model referred to as AFM-3D. The two-stage AFM-3D operates by initially predicting 3-D cube proposals, followed by generating refined 3-D prismatic models (LoD1 models) for each proposal. Thorough experimentation across different datasets demonstrates the superior performance of AFM and AFM-3D. A significant enhancement of 6.4% quality score is observed on the urban 3-D dataset in comparison to recent methods. In addition to the proposed novel methodology, we compare anchor-based and anchor-free bounding box generation mechanisms for remote sensing data, explore pixel-based and contour-based segmentation strategies, evaluate learning-based and empirical height estimation methods, and discuss the indispensability of DSM data in 3-D building instance extraction. These analyses yield valuable insights that contribute to the progression of 3-D building extraction research.

HigherNet-DST: Higher-Resolution Network With Dynamic Scale Training for Rooftop Delineation

High Spatial-Resolution Classification of Urban Surfaces Using a Deep Learning Method

HD-Net: High-resolution decoupled network for building footprint extraction via deeply supervised body and boundary decomposition

Deep Roof Refiner: A detail-oriented deep learning network for refined delineation of roof structure lines using satellite imagery

A Boundary Regulated Network for Accurate Roof Segmentation and Outline Extraction

Hi-ResNet: Edge Detail Enhancement for High-Resolution Remote Sensing Segmentation

Simultaneous Extraction of Spatial and Attributional Building Information Across Large-Scale Urban Landscapes from High-Resolution Satellite Imagery

Building Precision: Efficient Encoder–Decoder Networks for Remote Sensing Based on Aerial RGB and LiDAR Data

3-D Building Instance Extraction From High-Resolution Remote Sensing Images and DSM With an End-to-End Deep Neural Network

A Multi-Scale Mapping Approach Based on a Deep Learning CNN Model for Reconstructing High-Resolution Urban DEMs

HGDNet: A Height-Hierarchy Guided Dual-Decoder Network for Single View Building Extraction and Height Estimation

Multi-task deep learning for large-scale building detail extraction from high-resolution satellite imagery

HeightMapNet: Explicit Height Modeling for End-to-End HD Map Learning

OEC-RNN: Object-Oriented Delineation of Rooftops With Edges and Corners Using the Recurrent Neural Network From the Aerial Images

High-Resolution Terrain Modeling Using Airborne LiDAR Data with Transfer Learning

Multi-scale attention integrated hierarchical networks for high-resolution building footprint extraction

Deep Learning Guided Building Reconstruction from Satellite Imagery-derived Point Clouds

A Research on 3D Reconstruction of Building Rooftop Models from LiDAR Data and Orthophoto.

Sequentially Delineation of Rooftops with Holes from VHR Aerial Images Using a Convolutional Recurrent Neural Network

UB-FineNet: Urban Building Fine-grained Classification Network for Open-access Satellite Images