Abstract:Retrieval approaches that score documents based on learned dense vectors (i.e., dense retrieval) rather than lexical signals (i.e., conventional retrieval) are increasingly popular. Their ability to identify related documents that do not necessarily contain the same terms as those appearing in the user's query (thereby improving recall) is one of their key advantages. However, to actually achieve these gains, dense retrieval approaches typically require an exhaustive search over the document collection, making them considerably more expensive at query-time than conventional lexical approaches. Several techniques aim to reduce this computational overhead by approximating the results of a full dense retriever. Although these approaches reasonably approximate the top results, they suffer in terms of recall -- one of the key advantages of dense retrieval. We introduce 'LADR' (Lexically-Accelerated Dense Retrieval), a simple-yet-effective approach that improves the efficiency of existing dense retrieval models without compromising on retrieval effectiveness. LADR uses lexical retrieval techniques to seed a dense retrieval exploration that uses a document proximity graph. We explore two variants of LADR: a proactive approach that expands the search space to the neighbors of all seed documents, and an adaptive approach that selectively searches the documents with the highest estimated relevance in an iterative fashion. Through extensive experiments across a variety of dense retrieval models, we find that LADR establishes a new dense retrieval effectiveness-efficiency Pareto frontier among approximate k nearest neighbor techniques. Further, we find that when tuned to take around 8ms per query in retrieval latency on our hardware, LADR consistently achieves both precision and recall that are on par with an exhaustive search on standard benchmarks.

LED: Lexicon-Enlightened Dense Retriever for Large-Scale Retrieval.

Learning To Retrieve: How to Train a Dense Retrieval Model Effectively and Efficiently

A Multi-level Distillation based Dense Passage Retrieval Model

ControlRetriever: Harnessing the Power of Instructions for Controllable Retrieval

UnifieR: A Unified Retriever for Large-Scale Retrieval

SPLADE: Sparse Lexical and Expansion Model for First Stage Ranking

LEAD: Liberal Feature-based Distillation for Dense Retrieval

Large Language Models as Foundations for Next-Gen Dense Retrieval: A Comprehensive Empirical Assessment

Sparse, Dense, and Attentional Representations for Text Retrieval

LexMAE: Lexicon-Bottlenecked Pretraining for Large-Scale Retrieval

ERDL: Efficient Retrieval Framework Based on Distillation from Large Language Models

Towards Competitive Search Relevance For Inference-Free Learned Sparse Retrievers

BERM: Training the Balanced and Extractable Representation for Matching to Improve Generalization Ability of Dense Retrieval

Leveraging LLMs for Unsupervised Dense Retriever Ranking

Unsupervised Dense Retrieval with Relevance-Aware Contrastive Pre-Training

Improving Embedding-based Large-scale Retrieval Via Label Enhancement.

Lexically-Accelerated Dense Retrieval

DREditor: An Time-efficient Approach for Building a Domain-specific Dense Retrieval Model

Longtriever: a Pre-trained Long Text Encoder for Dense Document Retrieval

Learning Domain‐specific Semantic Representation from Weakly Supervised Data to Improve Research Dataset Retrieval

Dense X Retrieval: What Retrieval Granularity Should We Use?