Abstract:The initial retrieval stage of information retrieval aims to generate as many relevant candidate documents as possible in a simple yet efficient way. Traditional term based retrieval methods like BM25 deal with the problem based on Bag-of-Words (BoW) representation, thus they only focus on exact matching (i.e., syntactic) and lack the consideration for semantically related words. That causes the typical vocabulary mismatch problem and the reduction of performance in terms of recall. The advance of distributed representation (i.e., embedding) of words and documents provides an efficient way to measure the semantic relevance between words. Since embedding can alleviate the vocabulary mismatch problem, it is suitable for the initial retrieval task. We conduct several experiments to compare term basedmodels with embedding basedmodels in terms of recall. We compare above two branches of the initial retrieval models on three representative retrieval tasks (Web-QA, Ad-hoc retrieval and CQA respectively). The results show that embedding based method and term based method are complementary for each other and higher recall can be achieved by combining the above two types of models based on scores or ranking position. We find that combination of the two types of the models based on ranking position usually perform better than combination based on score. Furthermore, since queries and documents are in different forms for diverse application scenarios, it can be observed that the relative performance of the two types are almost same but the absolute performance are significant different regarding to distinct scenarios.

A Term Co-Occurrence Algorithm and the Effect of Co-Occurrence Terms on Result Ranking for Information Retrieval

A LDA Topic Model Based Collection Selection Method for Distributed Information Retrieval

Exploiting Community Feedback for Information Retrieval in Dht Networks

DF or IDF? On the Use of Primary Feature Model for Web Information Retrieval

Pseudo-Relevance Feedback Based On Mrmr Criteria

A Learning-Based Term-Weighting Approach for Information Retrieval

MFCRank: A Web Ranking Algorithm Based on Correlation of Multiple Features

Improved Combination of Multiple Retrieval Systems Using a Dynamic Combinatorial Fusion Algorithm.

Combining Multiple Retrieval Systems Using Combinatorial Fusion Analysis and Rank-Score Characteristic Function

Simple May Be Best - A Simple and Effective Method for Federated Web Search Via Search Engine Impact Factor Estimation.

Improve Precategorized Collection Retrieval by Using Supervised Term Weighting Schemes

A Study on Query Terms Proximity Embedding for Information Retrieval

A Comparison Between Term-Based and Embedding-Based Methods for Initial Retrieval

ISTC: A New Method for Clustering Search Results

Quotient Space-Based Keyword Retrieval in Sponsored Search

A Relevance Evaluation Model of Information Retrieval Based on Weighted Term Frequency

A Semantic Similarity Weighted Query Term Proximity Framework for Information Retrieval

Effects of Distance Information Between the Term and the Central Term on the Similar Question Matching

Combination of Multiple Retrieval Systems Using Rank-Score Function and Cognitive Diversity

Efficiently answering top-k frequent term queries in temporal-categorical range

Recency Ranking by Diversification of Result Set