Abstract:3-D building models play a vital role in numerous applications including urban planning and smart cities. Recent 3-D building modeling methods either rely heavily on available manually collected footprint references or hardly reach real automation on par with manual editing. To approach the automated extraction of instance-level 3-D buildings at level of detail 1 (LoD1), we introduce an innovative end-to-end 3-D building instance segmentation model. This model predicts accurate contours and heights of individual buildings simultaneously using ortho-rectified high-resolution remote sensing images and digital surface models (DSMs), getting rid of additional reference data and empirical parameter settings. First, we propose an anchor-free multihead (AFM) building extraction network tailored for extracting 2-D building contours. AFM incorporates a full-resolution, long-range correlation boosted global mask prediction branch along with anchor-free bounding box generation, as well as a newly developed online hard sample mining (OHSM) training procedure based on uncertainty analysis to emphasize error-prone positions in locating building contours. Subsequently, we incorporate a height prediction component to AFM in order to derive accurate building height information, thus creating the comprehensive 3-D building extraction model referred to as AFM-3D. The two-stage AFM-3D operates by initially predicting 3-D cube proposals, followed by generating refined 3-D prismatic models (LoD1 models) for each proposal. Thorough experimentation across different datasets demonstrates the superior performance of AFM and AFM-3D. A significant enhancement of 6.4% quality score is observed on the urban 3-D dataset in comparison to recent methods. In addition to the proposed novel methodology, we compare anchor-based and anchor-free bounding box generation mechanisms for remote sensing data, explore pixel-based and contour-based segmentation strategies, evaluate learning-based and empirical height estimation methods, and discuss the indispensability of DSM data in 3-D building instance extraction. These analyses yield valuable insights that contribute to the progression of 3-D building extraction research.

Srbuildingseg-e<sup>2</sup>: an integrated model for end-to-end higher-resolution building extraction

Making Low-Resolution Satellite Images Reborn: A Deep Learning Approach for Super-Resolution Building Extraction

Multi-scale Edge Constrained for Building Semantic Segmentation Using High Resolution Remote Sensing Images

An Automatic Building Extraction Method from High Resolution Satellite Image

HCRB-MSAN: Horizontally Connected Residual Blocks-Based Multiscale Attention Network for Semantic Segmentation of Buildings in HSR Remote Sensing Images

Building Extraction From High Spatial Resolution Remote Sensing Images of Complex Scenes by Combining Region-Line Feature Fusion and OCNN

EA U2-Net: An Efficient Building Extraction Algorithm Based on Complex Background Information

Automatic Building Extraction from Very High Resolution Satellite Imagery Using Line Segment Detector.

3-D Building Instance Extraction From High-Resolution Remote Sensing Images and DSM With an End-to-End Deep Neural Network

SNLRUX++ for Building Extraction From High-Resolution Remote Sensing Images

Building Extraction From Very High-Resolution Remote Sensing Images Using Refine-UNet

A Building Extraction Method for High-Resolution Remote Sensing Images with Multiple Attentions and Parallel Encoders Combining Enhanced Spectral Information

CSA-Net: Complex Scenarios Adaptive Network for Building Extraction for Remote Sensing Images

Automated Building Extraction Using Satellite Remote Sensing Imagery

RSBuilding: Towards General Remote Sensing Image Building Extraction and Change Detection with Foundation Model

Building Outline Extraction Directly Using the U2-Net Semantic Segmentation Model from High-Resolution Aerial Images and a Comparison Study

Building Extraction from Very-High-Resolution Remote Sensing Images Using Semi-Supervised Semantic Edge Detection

Enhancing Building Semantic Segmentation Accuracy with Super Resolution and Deep Learning: Investigating the Impact of Spatial Resolution on Various Datasets

Data Fusion of High-Resolution Satellite Imagery and GIS Data for Automatic Building Extraction

Extracting Buildings from Remote Sensing Images Using a Multitask Encoder-Decoder Network with Boundary Refinement

Segment Any Building