Abstract:Web search provides a promising way for people to obtain information and has been extensively studied. With the surge of deep learning and large-scale pre-training techniques, various neural information retrieval models are proposed, and they have demonstrated the power for improving search (especially, the ranking) quality. All these existing search methods follow a common paradigm, i.e., index-retrieve-rerank, where they first build an index of all documents based on document terms (i.e., sparse inverted index) or representation vectors (i.e., dense vector index), then retrieve and rerank retrieved documents based on the similarity between the query and documents via ranking models. In this paper, we explore a new paradigm of information retrieval without an explicit index but only with a pre-trained model. Instead, all of the knowledge of the documents is encoded into model parameters, which can be regarded as a differentiable indexer and optimized in an end-to-end manner. Specifically, we propose a pre-trained model-based information retrieval (IR) system called DynamicRetriever, which directly returns document identifiers for a given query. Under such a framework, we implement two variants to explore how to train the model from scratch and how to combine the advantages of dense retrieval models. Compared with existing search methods, the model-based IR system parameterizes the traditional static index with a pre-training model, which converts the document semantic mapping into a dynamic and updatable process. Extensive experiments conducted on the public search benchmark Microsoft machine reading comprehension (MS MARCO) verify the effectiveness and potential of our proposed new paradigm for information retrieval.

IncDSI: Incrementally Updatable Document Retrieval

De-DSI: Decentralised Differentiable Search Index

PromptDSI: Prompt-based Rehearsal-free Instance-wise Incremental Learning for Document Retrieval

Semantic-Enhanced Differentiable Search Index Inspired by Learning Strategies

Bridging the Gap Between Indexing and Retrieval for Differentiable Search Index with Query Generation

Understanding Differential Search Index for Text Retrieval

A Neural Corpus Indexer for Document Retrieval

Auto Search Indexer for End-to-End Document Retrieval

NOVO: Learnable and Interpretable Document Identifiers for Model-Based IR

DeeperImpact: Optimizing Sparse Learned Index Structures

DynamicRetriever: A Pre-trained Model-based IR System Without an Explicit Index

Transformer Memory as a Differentiable Search Index

An Incremental Learning Strategy for Search Results Optimization

EHI: End-to-end Learning of Hierarchical Index for Efficient Dense Retrieval

Model-enhanced Vector Index

Incrementally-Computable Neural Networks: Efficient Inference for Dynamic Inputs

A Scalable Learned Index Scheme in Storage Systems

DynamicRetriever: A Pre-training Model-based IR System with Neither Sparse nor Dense Index

Progressively Optimized Bi-Granular Document Representation for Scalable Embedding Based Retrieval

Neural-DINF: A Neural Network Based Framework for Measuring Document Influence

Efficient Document Ranking with Learnable Late Interactions