Abstract:Conventional evaluation of information retrieval (IR) systems is conducted empirically. Common metrics such as precision and recall are good performance indicators yet they fail to evaluate the functionality of the underlying IR models. Recently, inductive evaluation approaches have been proposed to circumvent this problem. In this thesis, we propose an “aboutness” based inductive evaluation for functional benchmarking of IR models. It is complementary to the traditional experimental based performance benchmarking (e.g. TREC). IR is driven by a process which determines the aboutness relationship between a document and a query. Recent attempts spawned from logic-based information retrieval theory have been made to formalize aboutness properties. Different IR models support different sets of aboutness properties. These sets enable the comparison of functionalities of various IR models from a theoretical viewpoint. Nonetheless, existing aboutness frameworks are ineffective for functional benchmarking. The proposed aboutness properties are largely determined by the framework, where the aboutness model is defined. Further, some aboutness properties are only sound within the context of a given IR model, yet they may not be sound from the perspective of the user. To overcome these predicaments, we investigate aboutness from a commonsense point of view, i.e. what are the fundamental aboutness properties acceptable from a human reasoning perspective, independent of any given IR model. We first adopt the most fundamental aboutness framework, namely the one proposed by Bruza, as the basis in defining initial functional benchmarking suite. We apply the suite to various IR models. We then re-assess the effectiveness of Bruza's framework for benchmarking and observe the advantages and deficiencies of the IR models under examination. Based on these observations and reassessments, we refine the existing notion of aboutness and propose a new commonsense aboutness theory. To illustrate the potential of this new theory, we apply it to functional benchmarking. The results can suggest how an IR model be improved to support desirable commonsense aboutness properties. The commonsense aboutness theory would lay down a significant theoretical groundwork in IR research, which could (if properly deployed) lead to more effective IR systems.

A Quasi-Current Representation for Information Needs Inspired by Two-State Vector Formalism

QIRM: A quantum interactive retrieval model for session search.

A Quantum Query Expansion Approach for Session Search.

How Quantum Theory is Developing the Field of Information Retrieval

Modeling Relevance Judgement Inspired by Quantum Weak Measurement

A Quantum Expectation Value Based Language Model with Application to Question Answering

Tensor Field Model for Higher-Order Information Retrieval.

Quantum-Inspired Neural Language Representation, Matching and Understanding.

An Adaptive Contextual Quantum Language Model

Exploring a Multidimensional Representation of Documents and Queries (extended version)

Quantum Theory-Inspired Search

Query Representation by Structured Concept Threads with Application to Interactive Video Retrieval.

Meaning-focused and Quantum-inspired Information Retrieval

A Novel Re-Ranking Approach Inspired By Quantum Measurement

Two-State Vector Formalism and Quantum Interference

Learning Informative Latent Representation for Quantum State Tomography

Exploration of Quantum Interference in Document Relevance Judgement Discrepancy

The Vector Representation of the Standard Quantum Process Tomography

Modeling Multidimensional User Relevance in IR using Vector Spaces

Modeling Quantum Entanglements in Quantum Language Models

A Commonsense Aboutness Theory for Information Retrieval Modeling