Abstract:In web image search, items users search for are images instead of Web pages or online services. Web image search constitutes a very important part of web search. Re-ranking is a trusted technique to improve retrieval effectiveness in web search. Previous work on re-ranking web image search results mainly focuses on intra-query information (e.g., human interactions with the initial list of the current query). Contextual information such as the query sequence and implicit user feedback provided during a search session prior to the current query is known to improve the performance of general web search but has so far not been used in web image search. The differences in result placement and interaction mechanisms of image search make the search process rather different from general Web search engines. Because of these differences, context-aware re-ranking models that have originally been developed for general web search cannot simply be applied to web image search. We propose CARM, a context-aware re-ranking model, a neural network-based framework to re-rank web image search results for a query based on previous interaction behavior in the search session in which the query was submitted. Specifically, we explore a hybrid encoder with an attention mechanism to model intra-query and inter-query user preferences for image results in a two-stage structure. We train context-aware re-ranking model (CARM) to jointly learn query and image representations so as to be able to deal with the multimodal characteristics of web image search. Extensive experiments are carried out on a commercial web image search dataset. The results show that CARM outperforms state-of-the-art baseline models in terms of personalized evaluation metrics. Also, CARM combines the original ranking can improve the original ranking on personalized ranking and relevance estimation. We make the implementation of CARM and relevant datasets publicly available to facilitate future studies.

Search-based relevance association with auxiliary contextual cues.

Image Retrieval by Cross-Media Relevance Fusion.

Contextual Modeling on Auxiliary Points for Robust Image Reranking

Cross-media relevance mining for evaluating text-based image search engine

Image Relevance Prediction Using Query-Context Bag-of-Object Retrieval Model

Collaborative Image Relevance Learning for Visual Re-Ranking.

Hierarchical Gumbel Attention Network for Text-based Person Search

An output aggregation system for large scale cross-modal retrieval

Image Search Reranking with Query-Dependent Click-Based Relevance Feedback.

Improving Web Image Search with Contextual Information.

Sparse Contextual Activation for Efficient Visual Re-Ranking

Image search by graph-based label propagation with image representation from DNN.

Click-through-based Subspace Learning for Image Search

Click-Through-Based Cross-View Learning For Image Search

Context-Aware Ranking In Web Search

Improving Search Relevance for Short Queries in Community Question Answering

Robust Interaction-Based Relevance Modeling for Online e-Commerce Search

Contextual Similarity Aggregation with Self-attention for Visual Re-ranking

Towards Context-Aware Evaluation for Image Search

Contextual image search.

Context‐aware relation enhancement and similarity reasoning for image‐text retrieval