Abstract:ABSTRACT Remote sensing image retrieval is to find the most identical or similar images to a query image in the vast archive of remote sensing images. A key process is to extract the most distinctive features. In this study, we introduce a second-order pooling named compact bilinear pooling (CBP) into convolutional neural networks (CNNs) for remote sensing image retrieval. The retrieval algorithm has three stages, pretraining, fine-tuning and retrieval. In the pretraining stage, two classic CNN structures, VGG16 and ResNet34, are pretrained respectively with the ImageNet consisting of close-range images. A CBP layer is introduced before the fully connected layers in the two networks. To extract globally consistent representations, a channel and spatial integrated attention mechanism is proposed to refine features from the last convolution layer and the features are used as the input of the CBP. In the fine-tuning stage, the new network is fine-tuned on a remote sensing dataset to train discriminable features. In the retrieval stage, the network, with fully connected layers being replaced by a PCA (principal component analysis) module, is applied to new remote sensing datasets. Our retrieval algorithm with the combination of CBP and PCA obtained the best performance and outperformed several mainstream pooling or encoding methods such as full-connected layer, IFK (Improved Fisher Kernel), BoW (Bag-of-Words) and maxpooling, etc. The channel and spatial attention mechanism contributes to the CBP based retrieval method and obtained the best performance on all the datasets, as well as outperformed several recent attention methods. Source code is available at http://study.rsgis/whu.edu.cn/pages/download.

Multi-scale Orderless Cross-Regions-pooling of Deep Attributes for Image Retrieval

Exploiting Hierarchical Activations of Neural Network for Image Retrieval.

Improving Cross-Dimensional Weighting Pooling with Multi-Scale Feature Fusion for Image Retrieval.

Deep Attributes from Context-Aware Regional Neural Codes

Two-stage Pooling of Deep Convolutional Features for Image Retrieval.

Image retrieval based on aggregated deep features weighted by regional significance and channel sensitivity

Cross-convolutional-layer Pooling for Generic Visual Recognition.

Deep Cross-Dimensional Attention Hashing for Image Retrieval.

Region-based Cascade Pooling of Convolutional Features for HRRS Image Retrieval

Attention Boosted Bilinear Pooling for Remote Sensing Image Retrieval

Aggregating Deep Convolutional Features for Image Retrieval Using Multi-regional Cross Weighting

Beyond Object Proposals: Random Crop Pooling for Multi-label Image Recognition

Learning Representation of Multi-Scale Object for Fine-Grained Image Retrieval.

FRWCAE: Joint Faster-Rcnn and Wasserstein Convolutional Auto-Encoder for Instance Retrieval

Augmented Image Retrieval Using Multi-Order Object Layout With Attributes

Multi-scale Gem Pooling with N-Pair Center Loss for Fine-Grained Image Search

Multi-level Image Representation for Large-Scale Image-Based Instance Retrieval.

Multi-view Rank Pooling for 3D Object Recognition*.

Deep Aggregation of Regional Convolutional Activations for Content Based Image Retrieval

Multilayer Convolutional Feature Aggregation Algorithm for Image Retrieval