Abstract:Relevance plays a central role in information retrieval (IR), which has received extensive studies starting from the 20th century. The definition and the modeling of relevance has always been critical challenges in both information science and computer science research areas. Along with the debate and exploration on relevance, IR has already become a core task in many real-world applications, such as Web search engines, question answering systems, conversational bots, and so on. While relevance acts as a unified concept in all these retrieval tasks, the inherent definitions are quite different due to the heterogeneity of these tasks. This raises a question to us: Do these different forms of relevance really lead to different modeling focuses? To answer this question, in this work, we conduct an empirical study on relevance modeling in three representative IR tasks, i.e., document retrieval, answer retrieval, and response retrieval. Specifically, we attempt to study the following two questions: 1) Does relevance modeling in these tasks really show differences in terms of natural language understanding (NLU)? We employ 16 linguistic tasks to probe a unified retrieval model over these three retrieval tasks to answer this question. 2) If there do exist differences, how can we leverage the findings to enhance the relevance modeling? We proposed three intervention methods to investigate how to leverage different modeling focuses of relevance to improve these IR tasks. We believe the way we study the problem as well as our findings would be beneficial to the IR community.

Concepts,Types,and Uncertainty of "Relevance"in Information Retrieval

On Relevance in Information Retrieval Based on Relevance Theory

On Relevance in Information Retrieval Based on Relevance Theory——information production,indexing

User-Oriented Relevance Judgment: A Conceptual Model

Relevance Judgment: What Do Information Users Consider Beyond Topicality?

A Linguistic Study on Relevance Modeling in Information Retrieval

Relevance Judgment in Epistemic and Hedonic Information Searches

When does Relevance Mean Usefulness and User Satisfaction in Web Search?

Understanding Relevance Judgments in Legal Case Retrieval

Research of Relevance Feedback Technology in Personalized Information Retrieval

Investigating Passage-level Relevance and Its Role in Document-level Relevance Judgment

Relevance in E-service Searching

Tentative Survey on Unboundlessness of Relevance Theory

Review on Relevance Criteria(2000-2010)

A Relevance Evaluation Model of Information Retrieval Based on Weighted Term Frequency

Explanation of Relevance Judgement Discrepancy with Quantum Interference.

A Document Relevance Based Search Result Re-Ranking

STATE OF THE ART OF INFORMATION RETRIEVAL ON THE WEB

A systematic review of multidimensional relevance estimation in information retrieval

User＇s Relevance of PIR System Based on Cloud Models

Relevance Estimation with Multiple Information Sources on Search Engine Result Pages.