Abstract:Relevance plays a central role in information retrieval (IR), which has received extensive studies starting from the 20th century. The definition and the modeling of relevance has always been critical challenges in both information science and computer science research areas. Along with the debate and exploration on relevance, IR has already become a core task in many real-world applications, such as Web search engines, question answering systems, conversational bots, and so on. While relevance acts as a unified concept in all these retrieval tasks, the inherent definitions are quite different due to the heterogeneity of these tasks. This raises a question to us: Do these different forms of relevance really lead to different modeling focuses? To answer this question, in this work, we conduct an empirical study on relevance modeling in three representative IR tasks, i.e., document retrieval, answer retrieval, and response retrieval. Specifically, we attempt to study the following two questions: 1) Does relevance modeling in these tasks really show differences in terms of natural language understanding (NLU)? We employ 16 linguistic tasks to probe a unified retrieval model over these three retrieval tasks to answer this question. 2) If there do exist differences, how can we leverage the findings to enhance the relevance modeling? We proposed three intervention methods to investigate how to leverage different modeling focuses of relevance to improve these IR tasks. We believe the way we study the problem as well as our findings would be beneficial to the IR community.

Empirical Analysis on Relevance Criteria Oriented Academic Information Retrieval System Success Model

Investigation and Analysis of Academic Information Retrieval System Based on Electronic Resources of Nanjing Uuniversity Library

On Relevance in Information Retrieval Based on Relevance Theory

A Linguistic Study on Relevance Modeling in Information Retrieval

A Framework for Scientific Paper Retrieval and Recommender Systems

Science Models as Value-Added Services for Scholarly Information Systems

Understanding the Role of Human-Inspired Heuristics for Retrieval Models

An information retrieval system based on a user profile

Research on Chinese Opinion Retrieval Based on Relevance Model

Analyzing the Quality of Academic Information Systems on System Success

Review on Relevance Criteria(2000-2010)

Model Construction and Empirical Research on Information Searching Behavior of Researchers from the Perspective of Situational Cognition

User-Oriented Relevance Judgment: A Conceptual Model

A Model of Personalized Information Retrieval Systems for Internet Applications

A Thematic Study of Requirements Modeling and Analysis for Self-Adaptive Systems

An Exploratory Study on Factors to Information Literacy： A Structural Equation Model Analyzing

A Systematic Literature Review of Requirements Modeling and Analysis for Self-adaptive Systems.

A Study of Academic Search Scenarios and Information Seeking Behaviour

EMPIRICAL RESEARCH ON SUCCESS CRITERIA OF IS/IT PROJECTS

A Relevance Evaluation Model of Information Retrieval Based on Weighted Term Frequency

Understanding Relevance Judgments in Legal Case Retrieval