Abstract:In content-based image retrieval (CBIR), one of the most challenging and ambiguous tasks is to correctly understand the human query intention and measure its semantic relevance with images in the database. Due to the impressive capability of visual saliency in predicting human visual attention that is closely related to the query intention, this paper attempts to explicitly discover the essential effect of visual saliency in CBIR via qualitative and quantitative experiments. Toward this end, we first generate the fixation density maps of images from a widely used CBIR dataset by using an eye-tracking apparatus. These ground-truth saliency maps are then used to measure the influence of visual saliency to the task of CBIR by exploring several probable ways of incorporating such saliency cues into the retrieval process. We find that visual saliency is indeed beneficial to the CBIR task, and the best saliency involving scheme is possibly different for different image retrieval models. Inspired by the findings, this paper presents two-stream attentive convolutional neural networks (CNNs) with saliency embedded inside for CBIR. The proposed network has two streams that simultaneously handle two tasks. The main stream focuses on extracting discriminative visual features that are tightly related to semantic attributes. Meanwhile, the auxiliary stream aims to facilitate the main stream by redirecting the feature extraction to the salient image content that a human may pay attention to. By fusing these two streams into the Main and Auxiliary CNNs (MAC), image similarity can be computed as the human being does by reserving conspicuous content and suppressing irrelevant regions. Extensive experiments show that the proposed model achieves impressive performance in image retrieval on four public datasets.

Bio-inspired visual saliency detection and its application on image retargeting

Color boosted visual saliency detection and its application to image classification

Image Retargeting Assessment Based on Salient Region Similarity

Detect saliency to understand a photo

Context-Aware Saliency Detection for Image Retargeting Using Convolutional Neural Networks

A biologically inspired computational model for image saliency detection.

Salient Region Detection by Fusing Foreground and Background Cues Extracted from Single Image

Visual saliency detection by integrating spatial position prior of object with background cues

Non-redundancy saliency map for image retargeting

Saliency-based visualization for image search

A Hybrid of Local and Global Saliencies for Detecting Image Salient Region and Appearance

Person Re-Identification Based on Visual Saliency

An Approach for Image Retrieval Based on Visual Saliency

A Bioinspired Visual Saliency Model

Bio-driven visual saliency detection with color factor

Saliency detection using a central stimuli sensitivity based model

Saliency-driven Scaling Optimization for Image Retargeting

Beyond Visual Retargeting: A Feature Retargeting Approach for Visual Recognition and Its Applications.

Color Image Retrieval Based on Visual Saliency Map

Saliency Inside: Learning Attentive CNNs for Content-Based Image Retrieval

Saliency Detection by Selective Strategy for Salient Object Segmentation.