Abstract:Building extraction is significant in urban planning, economic evaluation, and driverless technology development. However, automatic building extraction from high spatial resolution remote sensing images has been a challenging task due to the various building shapes and colors, imaging conditions, and complex background objects. Current methods in building extraction are generally based on deep convolution networks, and they mostly use an encoder-decoder architecture, wherein detailed building features and small buildings are easily omitted in continuous convolution operations. Moreover, buildings with blurred boundaries are only completely extracted with difficulty. To meet these challenges, we propose a multi-task architecture of frequency-spatial learning Transformer to extract buildings from high spatial resolution remote sensing images. Different from current architecture, we designed a frequency-spatial learning module in the framework of multi-task to synthesize the multi-scale spatial features and frequency decomposition features of high-resolution image. Spiking convolution is proposed in this study to enhance the frequency features of buildings by mimicking the neural transmission in human brains. In this way, multi-scale building features can be better preserved and distinguished from background objects. Moreover, a masked-attention Transformer is adopted to improve multi-scale building mask prediction accuracy by synthesizing successive pixel-wise up-sampled feature maps. We also propose a strategy to evaluate the practical transferability of the proposed method by mimicking practical application cases through training and evaluating images with different spatial resolutions from different study areas and datasets. Experiments using five public building datasets (WHU-Building Satellite Dataset I, WHU-Building Satellite Dataset II, Massachusetts Buildings Dataset, Inria Aerial Image Dataset, xBD Building Dataset) demonstrate the strong potential applicability of our proposed method for practical application cases. Our method outperforms five recently proposed state-of-the-art semantic segmentation methods with 36.60% accuracy improvement on extracted buildings and approximately 53.55% recall progress in extracting small building instances. The implementation code will be released after the paper is published.

Extraction of building footprint using MASK-RCNN for high resolution aerial imagery

Application of mask R-CNN for building detection in UAV remote sensing images

Improved Anchor-Free Instance Segmentation for Building Extraction from High-Resolution Remote Sensing Images

BUILDING SEGMENTATION FROM AIRBORNE VHR IMAGES USING MASK R-CNN

Extracting Buildings from Remote Sensing Images Using a Multitask Encoder-Decoder Network with Boundary Refinement

Automatic Building Extraction from Google Earth Images under Complex Backgrounds Based on Deep Instance Segmentation Network

Building Extraction From High Spatial Resolution Remote Sensing Images of Complex Scenes by Combining Region-Line Feature Fusion and OCNN

Automatic building footprint extraction from very high-resolution imagery using deep learning techniques

BUILDING OUTLINE EXTRACTION FROM AERIAL IMAGES USING CONVOLUTIONAL NEURAL NETWORKS

Improved Building Extraction from Remotely Sensed Images by Integration of Encode–Decoder and Edge Enhancement Models

Boundary Regularized Building Footprint Extraction From Satellite Images Using Deep Neural Network

Deep cross-domain building extraction for selective depth estimation from oblique aerial imagery

Boundary Shape-Preserving Model for Building Mapping From High-Resolution Remote Sensing Images

Building Extraction from Very High Resolution Aerial Imagery Using Joint Attention Deep Neural Network

Urban building extraction from high-resolution remote sensing imagery based on multi-scale recurrent conditional generative adversarial network

Extraction of Buildings in Urban Area for Surface Area Assessment from Satellite Imagery based on Morphological Building Index using SVM Classifier

Remote sensing image building detection method based on Mask R-CNN

MSFTrans: a multi-task frequency-spatial learning transformer for building extraction from high spatial resolution remote sensing images

Multi-Level Perceptual Network for Urban Building Extraction from High-Resolution Remote Sensing Images

Effective Building Extraction From High-Resolution Remote Sensing Images With Multitask Driven Deep Neural Network

3-D Building Instance Extraction From High-Resolution Remote Sensing Images and DSM With an End-to-End Deep Neural Network