Abstract:The initial retrieval stage of information retrieval aims to generate as many relevant candidate documents as possible in a simple yet efficient way. Traditional term based retrieval methods like BM25 deal with the problem based on Bag-of-Words (BoW) representation, thus they only focus on exact matching (i.e., syntactic) and lack the consideration for semantically related words. That causes the typical vocabulary mismatch problem and the reduction of performance in terms of recall. The advance of distributed representation (i.e., embedding) of words and documents provides an efficient way to measure the semantic relevance between words. Since embedding can alleviate the vocabulary mismatch problem, it is suitable for the initial retrieval task. We conduct several experiments to compare term basedmodels with embedding basedmodels in terms of recall. We compare above two branches of the initial retrieval models on three representative retrieval tasks (Web-QA, Ad-hoc retrieval and CQA respectively). The results show that embedding based method and term based method are complementary for each other and higher recall can be achieved by combining the above two types of models based on scores or ranking position. We find that combination of the two types of the models based on ranking position usually perform better than combination based on score. Furthermore, since queries and documents are in different forms for diverse application scenarios, it can be observed that the relative performance of the two types are almost same but the absolute performance are significant different regarding to distinct scenarios.

Efficient term proximity search with term-pair indexes.

Can phrase indexing help to process non-phrase queries?

Exploring and Exploiting Proximity Statistic for Information Retrieval Model.

Aggregation-Aware Top-k Computation for Full-Text Search

Using Proximity in Query Focused Multi-document Extractive Summarization

High Precision Information Retrieval with Multi-Term Relation.

A Term Co-Occurrence Algorithm and the Effect of Co-Occurrence Terms on Result Ranking for Information Retrieval

Scalable Top-K Spatial Keyword Search

Efficiently answering top-k frequent term queries in temporal-categorical range

Proximity Full-Text Search with a Response Time Guarantee by Means of Additional Indexes

TSS: Efficient Term Set Search in Large Peer-to-Peer Textual Collections

Augmented Keyword Search on Spatial Entity Databases

Processing Spatial Keyword Query As a Top-K Aggregation Query

A Comparison Between Term-Based and Embedding-Based Methods for Initial Retrieval

Efficient Bulk Loading to Accelerate Spatial Keyword Queries

Improve Precategorized Collection Retrieval by Using Supervised Term Weighting Schemes

Compact Indexing and Judicious Searching for Billion-Scale Microblog Retrieval.

Processing Long Queries Against Short Text

Proximity full-text searches of frequently occurring words with a response time guarantee

Multi-term relations for information retrieval

Efficient Type-Ahead Search on Relational Data: a TASTIER Approach.