Abstract:Automated extraction of buildings from Earth observation (EO) data is important for various applications, including updating of maps, risk assessment, urban planning, and policy-making. Combining data from different sensors, such as high-resolution multispectral images (HRI) and light detection and ranging (LiDAR) data, has shown great potential in building extraction. Deep learning (DL) is increasingly used in multi-modal data fusion and urban object extraction. However, DL-based multi-modal fusion networks may under-perform due to insufficient learning of "joint features" from multiple sources and oversimplified approaches to fusing multi-modal features. Recently, a hybrid attention-aware fusion network (HAFNet) has been proposed for building extraction from a dataset, including co-located Very-High-Resolution (VHR) optical images and light detection and ranging (LiDAR) joint data. The system reported good performances thanks to the adaptivity of the attention mechanism to the features of the information content of the three streams but suffered from model over-parametrization, which inevitably leads to long training times and heavy computational load. In this paper, the authors propose a restructuring of the scheme, which involved replacing VGG-16-like encoders with the recently proposed EfficientNet, whose advantages counteract exactly the issues found with the HAFNet scheme. The novel configuration was tested on multiple benchmark datasets, reporting great improvements in terms of processing times, and also in terms of accuracy. The new scheme, called HAFNetE (HAFNet with EfficientNet integration), appears indeed capable of achieving good results with less parameters, translating into better computational efficiency. Based on these findings, we can conclude that, given the current advancements in single-thread schemes, the classical multi-thread HAFNet scheme could be effectively transformed by the HAFNetE scheme by replacing VGG-16 with EfficientNet blocks on each single thread. The remarkable reduction achieved in computational requirements moves the system one step closer to on-board implementation in a possible, future "urban mapping" satellite constellation.

A Hybrid Attention-Aware Fusion Network (HAFNet) for Building Extraction from High-Resolution Imagery and LiDAR Data

Integrating EfficientNet into an HAFNet Structure for Building Mapping in High-Resolution Optical Earth Observation Data

ACMFNet: Attention-Based Cross-Modal Fusion Network for Building Extraction of Remote Sensing Images

Building Extraction from Very High Resolution Aerial Imagery Using Joint Attention Deep Neural Network

Multi-Scale Feature Fusion Attention Network for Building Extraction in Remote Sensing Images

Hybrid Attention Cascaded U-net for Building Extraction from Aerial Images

B-FGC-Net: A Building Extraction Network from High Resolution Remote Sensing Imagery

Efficient Building Extraction for High Spatial Resolution Images Based on Dual Attention Network

Building Multi-Feature Fusion Refined Network for Building Extraction from High-Resolution Remote Sensing Images

HAFNet: Hierarchical Attentive Fusion Network for Multispectral Pedestrian Detection

Multisource Hyperspectral and LiDAR Data Fusion for Urban Land-Use Mapping based on a Modified Two-Branch Convolutional Neural Network

Building Detection from Panchromatic and Multispectral Images with Dual-Stream Asymmetric Fusion Networks

EMAFF-Net: an enhanced multi-scale attentive feature fusion network for building extraction from VHR remote sensing images

Multi-Scale Attention Network for Building Extraction from High-Resolution Remote Sensing Images

Hierarchical Disentangling Network for Building Extraction from Very High Resolution Optical Remote Sensing Imagery

Joint Classification of Hyperspectral and LiDAR Data Using Height Information Guided Hierarchical Fusion-and-Separation Network

MAFF-HRNet: Multi-Attention Feature Fusion HRNet for Building Segmentation in Remote Sensing Images

Cross Attention-Based Multi-Scale Convolutional Fusion Network for Hyperspectral and LiDAR Joint Classification

HFA-Net: High Frequency Attention Siamese Network for Building Change Detection in VHR Remote Sensing Images.

Hybrid FusionNet: A Hybrid Feature Fusion Framework for Multi-Source High-Resolution Remote Sensing Image Classification

CMGFNet: A deep cross-modal gated fusion network for building extraction from very high-resolution remote sensing images