Abstract:Web search heavily relies on click-through behavior as an essential feedback signal for performance evaluation and improvement. Traditionally, click is usually treated as a positive implicit feedback signal of relevance or usefulness, while non-click is regarded as a signal of irrelevance or uselessness. However, there are many cases where users satisfy their information need with the contents shown on the Search Engine Result Page (SERP). This raises the problem of measuring the usefulness of non-click results and modeling user satisfaction in such circumstances. For a long period, understanding non-click results is challenging owing to the lack of user interactions. In recent years, the rapid development of neuroimaging technologies constitutes a paradigm shift in various industries, e.g., search, entertainment, and education. Therefore, we benefit from these technologies and apply them to bridge the gap between the human mind and the external search system in non-click situations. To this end, we analyze the differences in brain signals between the examination of nonclick search results in different usefulness levels. Inspired by these differences, we conduct supervised learning tasks to estimate the usefulness of non-click results with brain signals and conventional information (i.e., content and context factors). Furthermore, we devise two re-ranking methods, i.e., a Personalized Method (PM) and a Generalized Intent modeling Method (GIM), for search result re-ranking with the estimated usefulness. Results show that it is feasible to utilize brain signals to improve usefulness estimation performance and enhance human-computer interactions by search result re-ranking.

When does Relevance Mean Usefulness and User Satisfaction in Web Search?

Investigating Result Usefulness in Mobile Search.

Understanding and Predicting Usefulness Judgment in Web Search

Relevance Estimation with Multiple Information Sources on Search Engine Result Pages.

Understanding Relevance Judgments in Legal Case Retrieval

A Document Relevance Based Search Result Re-Ranking

Relevance Judgment in Epistemic and Hedonic Information Searches

Usefulness as the Criterion for Evaluation of Interactive Information Retrieval

A Comparison of Retrieval Result Relevance Judgments Between American and Chinese Users

A Linguistic Study on Relevance Modeling in Information Retrieval

On Relevance in Information Retrieval Based on Relevance Theory

Search Result Reranking with Visual and Structure Information Sources

User-Oriented Relevance Judgment: A Conceptual Model

Predicting Relevance based on Assessor Disagreement: Analysis and Practical Applications for Search Evaluation

Investigating Passage-level Relevance and Its Role in Document-level Relevance Judgment

Ranking User's Relevance to a Topic Through Link Analysis on Web Logs

Meta-evaluation of Online and Offline Web Search Evaluation Metrics

Relevance Judgment: What Do Information Users Consider Beyond Topicality?

Automatic Search Engine Performance Evaluation With The Wisdom Of Crowds

Why Don't You Click: Understanding Non-Click Results inWeb Search with Brain Signals

Satisfaction Prediction of Web Search Users