Abstract:In image search re-ranking, besides the well-known semantic gap, intent gap, which is the gap between the representation of users' query/demand and the real intent of the users, is becoming a major problem restricting the development of image retrieval. To reduce human effects, in this paper, we use image click-through data, which can be viewed as the implicit feedback from users, to help overcome the intention gap, and further improve the image search performance. Generally, the hypothesis-visually similar images should be close in a ranking list-and the strategy-images with higher relevance should be ranked higher than others-are widely accepted. To obtain satisfying search results, thus, image similarity and the level of relevance typicality are determinate factors correspondingly. However, when measuring image similarity and typicality, conventional re-ranking approaches only consider visual information and initial ranks of images, while overlooking the influence of click-through data. This paper presents a novel re-ranking approach, named spectral clustering re-ranking with click-based similarity and typicality. First, to learn an appropriate similarity measurement, we propose click-based multi-feature similarity learning algorithm, which conducts metric learning based on click-based triplets selection, and integrates multiple features into a unified similarity space via multiple kernel learning. Then, based on the learnt click-based image similarity measure, we conduct spectral clustering to group visually and semantically similar images into same clusters, and get the final re-rank list by calculating click-based clusters typicality and within-clusters click-based image typicality in descending order. Our experiments conducted on two real-world query-image data sets with diverse representative queries show that our proposed re-ranking approach can significantly improve initial search results, and outperform several existing re-ranking approaches.

Efficient Image Reranking by Leveraging Click Data

Image Retrieval by Cross-Media Relevance Fusion.

Image Search Reranking with Query-Dependent Click-Based Relevance Feedback.

Cross-media relevance mining for evaluating text-based image search engine

Click-boosting multi-modality graph-based reranking for image search

Web Image Search Re-Ranking with Click-Based Similarity and Typicality

Image search by graph-based label propagation with image representation from DNN.

Learning to Rank Using User Clicks and Visual Features for Image Retrieval

Click-through-based Subspace Learning for Image Search

Click-boosting random walk for image search reranking

Click prediction for web image reranking using multimodal sparse coding.

Bag-Of-Words Based Deep Neural Network For Image Retrieval

An output aggregation system for large scale cross-modal retrieval

Scalable relevance feedback using click-through data for web image retrieval.

Click-Through-Based Cross-View Learning For Image Search

Search-based relevance association with auxiliary contextual cues.

USTC-CityU at MSR-Bing IRC: Image search by graph-based label propagation

Online Learning to Rank for Content-Based Image Retrieval.

Enhancing Content-Based Image Retrieval by Exploiting Relevance Feedback Logs

Learning to Combine Ad-hoc Ranking Functions for Image Retrieval

Using Implicit Relevane Feedback to Advance Web Image Search