Abstract:Web search heavily relies on click-through behavior as an essential feedback signal for performance improvement and evaluation. Traditionally, click is usually treated as a positive implicit feedback signal of relevance or usefulness, while non-click (especially non-click after examination) is regarded as a signal of irrelevance or uselessness. However, there are many cases where users do not click on any search results but still satisfy their information need with the contents of the results shown on the Search Engine Result Page (SERP). This raises the problem of measuring result usefulness and modeling user satisfaction in "Zero-click" search scenarios. Previous works have solved this issue by (1) detecting user satisfaction for abandoned SERP with context information and (2) considering result-level click necessity with external assessors' annotations. However, few works have investigated the reason behind non-click behavior and estimated the usefulness of non-click results. A challenge for this research question is how to collect valuable feedback for non-click results. With neuroimaging technologies, we design a lab-based user study and reveal differences in brain signals while examining non-click search results with different usefulness levels. The findings in significant brain regions and electroencephalogram~(EEG) spectrum also suggest that the process of usefulness judgment might involve similar cognitive functions of relevance perception and satisfaction decoding. Inspired by these findings, we conduct supervised learning tasks to estimate the usefulness of non-click results with brain signals and conventional information (i.e., content and context factors). Results show that it is feasible to utilize brain signals to improve usefulness estimation performance and enhancing human-computer interactions in "Zero-click" search scenarios.

SogouQ: The First Large-Scale Test Collection with Click Streams Used in a Shared-Task Evaluation

Sogou-ST: A New Dataset with Large-scale Refined Real-world Web Search Sessions

Sogou-QCL

SogouT-16

Tiangong-St: A New Dataset With Large-Scale Refined Real-World Web Search Sessions

Quda: Natural Language Queries for Visual Data Analytics

Clickage: towards bridging semantic and intent gaps via mining click logs of search engines.

TripClick: The Log Files of a Large Health Web Search Engine

Investigating Characteristics Of Non-Click Behavior Using Query Logs

Click-Feedback Retrieval

MS MARCO Web Search: a Large-scale Information-rich Web Dataset with Millions of Real Click Labels

Automatic Search Engine Performance Evaluation With The Wisdom Of Crowds

Towards Better Understanding of User Satisfaction in Open-Domain Conversational Search

A Large-Scale Study of Mobile Search Examination Behavior.

Why Don't You Click: Neural Correlates of Non-Click Behaviors in Web Search

A Study of User Image Search Behavior Based on Log Analysis.

The Task-oriented Queries Benchmark (ToQB)

Searching, browsing, and clicking in a search session: changes in user behavior by task and over time

Identifying queries in the wild, wild web.

SOPING: a Chinese customer review mining system.