Abstract:Compared to general web search engines, web image search engines display results in a different way. In web image search, results are typically placed in a grid-based manner rather than a sequential result list. In this scenario, users can view results not only in a vertical direction but also in a horizontal direction. Moreover, pagination is usually not (explicitly) supported on image search search engine result pages (SERPs), and users can view results by scrolling down without having to click a “next page” button. These differences lead to different interaction mechanisms and user behavior patterns, which, in turn, create challenges to evaluation metrics that have originally been developed for general web search. While considerable effort has been invested in developing evaluation metrics for general web search, there has been relatively little effort to construct grid-based evaluation metrics. To inform the development of grid-based evaluation metrics for web image search, we conduct a comprehensive analysis of user behavior so as to uncover how users allocate their attention in a grid-based web image search result interface. We obtain three findings: (1) “Middle bias”: Confirming previous studies, we find that image results in the horizontal middle positions may receive more attention from users than those in the leftmost or rightmost positions. (2) “Slower decay”: Unlike web search, users' attention does not decrease monotonically or dramatically with the rank position in image search, especially within a row. (3) “Row skipping”: Users may ignore particular rows and directly jump to results at some distance. Motivated by these observations, we propose corresponding user behavior assumptions to capture users' search interaction processes and evaluate their search performance. We show how to derive new metrics from these assumptions and demonstrate that they can be adopted to revise traditional list-based metrics like Discounted Cumulative Gain (DCG) and Rank-Biased Precision (RBP). To show the effectiveness of the proposed grid-based metrics, we compare them against a number of list-based metrics in terms of their correlation with user satisfaction. Our experimental results show that the proposed grid-based evaluation metrics better reflect user satisfaction in web image search.

Towards Context-Aware Evaluation for Image Search

Contextual Modeling on Auxiliary Points for Robust Image Reranking

Grid-based Evaluation Metrics for Web Image Search

On Annotation Methodologies for Image Search Evaluation

Preference-based Evaluation Metrics for Web Image Search

Using Context to Improve the Evaluation of Information Retrieval Systems

Contextual Hashing for Large-Scale Image Search.

How Well Do Offline and Online Evaluation Metrics Measure User Satisfaction in Web Image Search?

Contextual image search.

Improving Web Image Search with Contextual Information.

Community-Aware Photo Quality Evaluation by Deeply Encoding Human Perception

Contextual Query Expansion for Image Retrieval

CSIR4G: an Effective and Efficient Cross-Scenario Image Retrieval Model for Glasses

Contextual Visual Similarity

Research On Visual Performance Evaluation Model Of E-Commerce Websites

Exploration of Image Search Results Quality Assessment

Context‐aware relation enhancement and similarity reasoning for image‐text retrieval

Context-Based Adaptive Personalized Web Search for Improving Information Retrieval Effectiveness

Encoding Spatial Context for Large-Scale Partial-Duplicate Web Image Retrieval

Image Relevance Prediction Using Query-Context Bag-of-Object Retrieval Model

Cross-Media Similarity Evaluation for Web Image Retrieval in the Wild