Abstract:The dramatic growth in the number of application domains that naturally generate probabilistic, uncertain data has resulted in a need for efficiently supporting complex querying and decision-making over such data. In this paper, we address the problem of on-the-fly clustering and ranking over probabilistic databases. We begin with a systematic exploration of ranking in probabilistic databases by viewing it as a multi-criteria optimization problem, and by deriving a set of features that capture the key properties of a probabilistic dataset that dictate the ranked result. We contend that a single, specific ranking function may not suffice for probabilistic databasess, and we instead propose two parameterized ranking functions, called PRF w and PRF , that can approximate many of the previously proposed ranking functions. We present several novel algorithms for efficient computing such ranking functions using generating functions, even over databases that exhibit complex correlation patterns modeled using probabilistic and/xor trees or Markov networks. We further propose that the parameters of the ranking function be learned from user preferences, and develop an approach to learn such parameters. We also develop a hierarchical framework for efficiently combining on-the-fly clustering and ranking (called a ClusterRank query) over probabilistic databases. Our framework is based on a general definition of clustering, called restricted soft-t clustering, where a tuple is allowed to participate in at most t clusters. We show how several of our ranking functions can be seamlessly integrated into this framework, which not only allows ranking to continue in parallel with clustering, but also enables pruning of a large portion of the search space. Finally, we present a comprehensive experimental study comparing different ranking functions, and illustrating the effectiveness of our clustering framework.

A retrieval model family based on the probability ranking principle for ad hoc retrieval

Beyond Probability Ranking Principle: Modeling the Dependencies among Documents

A LDA Topic Model Based Collection Selection Method for Distributed Information Retrieval

Using the Dempster-Shafer Theory of Evidence to Rank Documents

The CLEO-III Ring Imaging Cherenkov Detector

Ranking Heterogeneous Search Result Pages using the Interactive Probability Ranking Principle

A Unified Approach to Ranking in Probabilistic Databases

ILCR: Item-based Latent Factors for Sparse Collaborative Retrieval

Probabilistic model for contextual retrieval.

Survey on Probability Models of Information Retrieval

A Modern Perspective on Query Likelihood with Deep Generative Retrieval Models

Not All Relevance Scores are Equal: Efficient Uncertainty and Calibration Modeling for Deep Retrieval Models

Enhancing the Ranking Context of Dense Retrieval Methods through Reciprocal Nearest Neighbors

Modeling Queries with Contextual Snippets for Information Retrieval.

A new probabilistic model for top-k ranking problem.

Reinforcement Learning to Rank with Markov Decision Process

On the Calibration and Uncertainty of Neural Learning to Rank Models

TPRF: A Transformer-based Pseudo-Relevance Feedback Model for Efficient and Effective Retrieval

Retrieval for Extremely Long Queries and Documents with RPRS: a Highly Efficient and Effective Transformer-based Re-Ranker

Ranking and Clustering in Probabilistic Databases

Quam: Adaptive Retrieval through Query Affinity Modelling