Abstract:The advancement in satellite image sensors has enabled the acquisition of high-resolution remote sensing (HRRS) images. However, interpreting these images accurately and obtaining the computational power needed to do so is challenging due to the complexity involved. This manuscript proposed a multi-stream convolutional neural network (CNN) fusion framework that involves multi-scale and multi-CNN integration for HRRS image recognition. The pre-trained CNNs were used to learn and extract semantic features from multi-scale HRRS images. Feature extraction using pre-trained CNNs is more efficient than training a CNN from scratch or fine-tuning a CNN. Discriminative canonical correlation analysis (DCCA) was used to fuse deep features extracted across CNNs and image scales. DCCA reduced the dimension of the features extracted from CNNs while providing a discriminative representation by maximizing the within-class correlation and minimizing the between-class correlation. The proposed model has been evaluated on NWPU-RESISC45 and UC Merced datasets. The accuracy associated with DCCA was 10% and 6% higher than discriminant correlation analysis (DCA) in the NWPU-RESISC45 and UC Merced datasets. The advantage of DCCA was better demonstrated in the NWPU-RESISC45 dataset due to the incorporation of richer within-class variability in this dataset. While both DCA and DCCA minimize between-class correlation, only DCCA maximizes the within-class correlation and, therefore, attains better accuracy. The proposed framework achieved higher accuracy than all state-of-the-art frameworks involving unsupervised learning and pre-trained CNNs and 2–3% higher than the majority of fine-tuned CNNs. The proposed framework offers computational time advantages, requiring only 13 s for training in NWPU-RESISC45, compared to a day for fine-tuning the existing CNNs. Thus, the proposed framework achieves a favourable balance between efficiency and accuracy in HRRS image recognition.

Two-stream feature aggregation deep neural network for scene classification of remote sensing images

Dense Connectivity Based Two-Stream Deep Feature Fusion Framework for Aerial Scene Classification

A lightweight and stochastic depth residual attention network for remote sensing scene classification

Deep Feature Fusion for High-Resolution Aerial Scene Classification

A Two-Stream Deep Fusion Framework for High-Resolution Aerial Scene Classification

Transferring Deep Convolutional Neural Networks for the Scene Classification of High-Resolution Remote Sensing Imagery

Multi-source remote sensing image classification based on two-channel densely connected convolutional networks

A Deep Neural Network Combined CNN and GCN for Remote Sensing Scene Classification

Superpixel-Based Long-Range Dependent Network for High-Resolution Remote-Sensing Image Classification

A Deep Neural Network Combined With Context Features for Remote Sensing Scene Classification

Remote Sensing Scene Classification by Gated Bidirectional Network

Deep Feature Fusion via Two-Stream Convolutional Neural Network for Hyperspectral Image Classification

Multi-Scale and Multi-Network Deep Feature Fusion for Discriminative Scene Classification of High-Resolution Remote Sensing Images

Deep Feature Aggregation Network for Hyperspectral Remote Sensing Image Classification

Looking Closer at the Scene: Multiscale Representation Learning for Remote Sensing Image Scene Classification

DS-HyFA-Net: A Deeply Supervised Hybrid Feature Aggregation Network With Multiencoders for Change Detection in High-Resolution Imagery

Dual-Stream Discriminative Attention Network for Cross-Scene Hyperspectral Image Classification

Remote Sensing Scene Classification Based on Multi-Structure Deep Features Fusion

Single-stream CNN with Learnable Architecture for Multi-source Remote Sensing Data.

Hyperspectral Image Spectral–Spatial Classification Method Based on Deep Adaptive Feature Fusion

Semantic Information Feature Aggregation Network for Object Detection in Remote Sensing Images