Abstract:The advancement in satellite image sensors has enabled the acquisition of high-resolution remote sensing (HRRS) images. However, interpreting these images accurately and obtaining the computational power needed to do so is challenging due to the complexity involved. This manuscript proposed a multi-stream convolutional neural network (CNN) fusion framework that involves multi-scale and multi-CNN integration for HRRS image recognition. The pre-trained CNNs were used to learn and extract semantic features from multi-scale HRRS images. Feature extraction using pre-trained CNNs is more efficient than training a CNN from scratch or fine-tuning a CNN. Discriminative canonical correlation analysis (DCCA) was used to fuse deep features extracted across CNNs and image scales. DCCA reduced the dimension of the features extracted from CNNs while providing a discriminative representation by maximizing the within-class correlation and minimizing the between-class correlation. The proposed model has been evaluated on NWPU-RESISC45 and UC Merced datasets. The accuracy associated with DCCA was 10% and 6% higher than discriminant correlation analysis (DCA) in the NWPU-RESISC45 and UC Merced datasets. The advantage of DCCA was better demonstrated in the NWPU-RESISC45 dataset due to the incorporation of richer within-class variability in this dataset. While both DCA and DCCA minimize between-class correlation, only DCCA maximizes the within-class correlation and, therefore, attains better accuracy. The proposed framework achieved higher accuracy than all state-of-the-art frameworks involving unsupervised learning and pre-trained CNNs and 2–3% higher than the majority of fine-tuned CNNs. The proposed framework offers computational time advantages, requiring only 13 s for training in NWPU-RESISC45, compared to a day for fine-tuning the existing CNNs. Thus, the proposed framework achieves a favourable balance between efficiency and accuracy in HRRS image recognition.

A Multi-Branch Feature Fusion Network for Building Detection in Remote Sensing Images

Dual-Branch Feature Fusion Network for Salient Object Detection

Multi-Scale Feature Fusion Attention Network for Building Extraction in Remote Sensing Images

Building Multi-Feature Fusion Refined Network for Building Extraction from High-Resolution Remote Sensing Images

Aerial Scene Classification Via Multilevel Fusion Based on Deep Convolutional Neural Networks.

A Dual-Branch Fusion Network Based on Reconstructed Transformer for Building Extraction in Remote Sensing Imagery

A Multi-Branch Feature Fusion Strategy Based on an Attention Mechanism for Remote Sensing Image Scene Classification

ACMFNet: Attention-Based Cross-Modal Fusion Network for Building Extraction of Remote Sensing Images

Cross-level and multiscale CNN-Transformer network for automatic building extraction from remote sensing imagery

Building Extraction From High Spatial Resolution Remote Sensing Images of Complex Scenes by Combining Region-Line Feature Fusion and OCNN

Multi-Scale and Multi-Network Deep Feature Fusion for Discriminative Scene Classification of High-Resolution Remote Sensing Images

Extracting Buildings from Remote Sensing Images Using a Multitask Encoder-Decoder Network with Boundary Refinement

Complementarity-aware Local-global Feature Fusion Network for Building Extraction in Remote Sensing Images

Multi-Feature Enhanced Building Change Detection Based on Semantic Information Guidance

Remote Sensing Scene Classification Based on Multi-Structure Deep Features Fusion

Hybrid transformer-CNN networks using superpixel segmentation for remote sensing building change detection

Densely Based Multi-Scale and Multi-Modal Fully Convolutional Networks for High-Resolution Remote-Sensing Image Semantic Segmentation

A cross-stage features fusion network for building extraction from remote sensing images

An Adaptive Attention Fusion Mechanism Convolutional Network for Object Detection in Remote Sensing Images

Object Detection in Remote Sensing Images Based on Adaptive Multi-Scale Feature Fusion Method

Building Extraction from Very High Resolution Aerial Imagery Using Joint Attention Deep Neural Network