Abstract:In content-based image retrieval (CBIR), one of the most challenging and ambiguous tasks is to correctly understand the human query intention and measure its semantic relevance with images in the database. Due to the impressive capability of visual saliency in predicting human visual attention that is closely related to the query intention, this paper attempts to explicitly discover the essential effect of visual saliency in CBIR via qualitative and quantitative experiments. Toward this end, we first generate the fixation density maps of images from a widely used CBIR dataset by using an eye-tracking apparatus. These ground-truth saliency maps are then used to measure the influence of visual saliency to the task of CBIR by exploring several probable ways of incorporating such saliency cues into the retrieval process. We find that visual saliency is indeed beneficial to the CBIR task, and the best saliency involving scheme is possibly different for different image retrieval models. Inspired by the findings, this paper presents two-stream attentive convolutional neural networks (CNNs) with saliency embedded inside for CBIR. The proposed network has two streams that simultaneously handle two tasks. The main stream focuses on extracting discriminative visual features that are tightly related to semantic attributes. Meanwhile, the auxiliary stream aims to facilitate the main stream by redirecting the feature extraction to the salient image content that a human may pay attention to. By fusing these two streams into the Main and Auxiliary CNNs (MAC), image similarity can be computed as the human being does by reserving conspicuous content and suppressing irrelevant regions. Extensive experiments show that the proposed model achieves impressive performance in image retrieval on four public datasets.

Scene Image Retrieval with Siamese Spatial Attention Pooling

Exploring Geometric Information in CNN for Image Retrieval.

SceneSketcher-v2: Fine-Grained Scene-Level Sketch-Based Image Retrieval Using Adaptive GCNs

Attention Boosted Bilinear Pooling for Remote Sensing Image Retrieval

Saliency Inside: Learning Attentive CNNs for Content-Based Image Retrieval

Hi-SIGIR: Hierachical Semantic-Guided Image-to-image Retrieval Via Scene Graph

Sketch-Based Image Retrieval Via Siamese Convolutional Neural Network

Content-based Image Retrieval Using Computational Visual Attention Model.

Content-based image retrieval using local visual attention feature

Image Retrieval Based on a Hybrid Model of Deep Convolutional Encoder

Successively focused details: attention-based recurrent multiscale network for sketch-based image retrieval

Attention-driven salient edge(s) and region(s) extraction with application to CBIR

Remote Sensing Image Retrieval Based on DenseNet Model and CBAM

Spatial-Content Image Search In Complex Scenes

End-to-end Semantic-Aware Object Retrieval Based on Region-Wise Attention

Exploring Spatial and Channel Contribution for Object Based Image Retrieval

Medical Image Retrieval Based on Convolutional Neural Network and Supervised Hashing

An Approach for Image Retrieval Based on Visual Saliency

Two-stage Content Based Image Retrieval Using Sparse Representation and Feature Fusion

Scene image retrieval via re-ranking semantic and packed dense interestpoints