Abstract:ABSTRACTLarge web search engines are facing formidable performance challenges because they have to process thousands of queries per second on tens of billions of documents, within interactive response time. Among many others, Top-k query processing (also called early termination or dynamic pruning) is an important class of optimization techniques that can improve the search efficiency and achieve faster query processing by avoiding the scoring of documents that are unlikely to be in the top results. One recent technique is using Block-Max index. In the Block-Max index, the posting lists are organized as blocks and the maximum score for each block is stored to improve the query efficiency. Although query processing speedup is achieved with Block-Max index, the ranking function for the Top-k results is the term-based approach. It is well known that documents' static scores are also important for a good ranking function. In this paper, we show that the performance of the state-of-the-art algorithms with the Block-Max index is degraded when the static score is added in the ranking function. Then we study efficient techniques for Top-k query processing in the case where a page's static score is given, such as PageRank, in addition to the term-based approach. In particular, we propose a set of new algorithms based on the WAND and MaxScore with Block-Max index using local score, which outperform the existing ones. Then we propose new techniques to estimate a better score upper bound for each block. We also study the search efficiency on different index structures where the document identifiers are assigned by URL sorting or by static document scores. Experiments on TREC GOV2 and ClueWeb09B show that considerable performance gains are achieved.

Durable top-k search in document archives

Durable Queries over Historical Time Series

Durable Queries over Non-synchronized Temporal Data

A Dual Hashtables Algorithm for Durable Top-k Search

Continuous Temporal Top-k Query over Versioned Documents.

Durable Reverse Top-K Queries on Time-Varying Preference

Scalable Top-K Spatial Keyword Search

Scalable Continual Top-k Keyword Search in Relational Databases

Ranking Archived Documents for Structured Queries on Semantic Layers

Processing Spatial Keyword Query As a Top-K Aggregation Query

Processing Long Queries Against Short Text

Anytime Ranking on Document-Ordered Indexes

Aggregation-Aware Top-k Computation for Full-Text Search

Ranked Document Retrieval in External Memory

Efficient Top K Temporal Spatial Keyword Search

Towards an Optimal Space-and-Query-Time Index for Top-k Document Retrieval

Optimized top-k processing with global page scores on block-max indexes.

Efficient Algorithms for Top-k Keyword Queries on Spatial Databases

Temporal queries and version management in XML-based document archives

Efficient Top-K Retrieval on Massive Data

KV-match: An Efficient Subsequence Matching Approach for Large Scale Time Series.