Abstract:The advancement in satellite image sensors has enabled the acquisition of high-resolution remote sensing (HRRS) images. However, interpreting these images accurately and obtaining the computational power needed to do so is challenging due to the complexity involved. This manuscript proposed a multi-stream convolutional neural network (CNN) fusion framework that involves multi-scale and multi-CNN integration for HRRS image recognition. The pre-trained CNNs were used to learn and extract semantic features from multi-scale HRRS images. Feature extraction using pre-trained CNNs is more efficient than training a CNN from scratch or fine-tuning a CNN. Discriminative canonical correlation analysis (DCCA) was used to fuse deep features extracted across CNNs and image scales. DCCA reduced the dimension of the features extracted from CNNs while providing a discriminative representation by maximizing the within-class correlation and minimizing the between-class correlation. The proposed model has been evaluated on NWPU-RESISC45 and UC Merced datasets. The accuracy associated with DCCA was 10% and 6% higher than discriminant correlation analysis (DCA) in the NWPU-RESISC45 and UC Merced datasets. The advantage of DCCA was better demonstrated in the NWPU-RESISC45 dataset due to the incorporation of richer within-class variability in this dataset. While both DCA and DCCA minimize between-class correlation, only DCCA maximizes the within-class correlation and, therefore, attains better accuracy. The proposed framework achieved higher accuracy than all state-of-the-art frameworks involving unsupervised learning and pre-trained CNNs and 2–3% higher than the majority of fine-tuned CNNs. The proposed framework offers computational time advantages, requiring only 13 s for training in NWPU-RESISC45, compared to a day for fine-tuning the existing CNNs. Thus, the proposed framework achieves a favourable balance between efficiency and accuracy in HRRS image recognition.

Surface tension of short flexible Lennard-Jones chains: Corresponding states behavior.

ObjectFusion: an Object Detection and Segmentation Framework with RGB-D SLAM and Convolutional Neural Networks

Dynamic Convolution Covariance Network Using Multi-Scale Feature Fusion for Remote Sensing Scene Image Classification

3D-SSD: Learning Hierarchical Features from RGB-D Images for Amodal 3D Object Detection

Large-Margin Multi-Modal Deep Learning for RGB-D Object Recognition

CNN-Based RGB-D Salient Object Detection: Learn, Select, and Fuse

Discriminative Cross-Modal Transfer Learning and Densely Cross-Level Feedback Fusion for RGB-D Salient Object Detection

Multimodal deep learning for robust RGB-D object recognition

MMSS: Multi-modal Sharable and Specific Feature Learning for RGB-D Object Recognition.

Structure-Aware Multimodal Feature Fusion for RGB-D Scene Classification and Beyond

Multi-Scale and Multi-Network Deep Feature Fusion for Discriminative Scene Classification of High-Resolution Remote Sensing Images

Weakly-supervised DCNN for RGB-D Object Recognition in Real-World Applications Which Lack Large-scale Annotated Training Data

RGB-T Salient Object Detection Via Fusing Multi-level CNN Features.

RGB-D-Based Object Recognition Using Multimodal Convolutional Neural Networks: A Survey

Modality And Component Aware Feature Fusion For Rgb-D Scene Classification

Multimodal Recurrent Neural Networks with Information Transfer Layers for Indoor Scene Labeling

Multi-Modality Task Cascade for 3D Object Detection

Spatial Context-Aware Object-Attentional Network for Multi-Label Image Classification

Multi-type and Multi-level Feature Fusion Network for RGBD Indoor Semantic Segmentation

CNNs-Based RGB-D Saliency Detection Via Cross-View Transfer and Multiview Fusion.

Discriminative feature fusion for RGB-D salient object detection