Abstract:Recently, generative retrieval has emerged as a promising alternative to the traditional retrieval paradigms. It assigns each document a unique identifier, known as the DocID, and employs a generative model to directly generate the relevant DocID for the input query. A common choice for the DocID is one or several natural language sequences, e.g. the title, synthetic queries, or n-grams, so that the pre-trained knowledge of the generative model can be effectively utilized. However, a sequence is generated token by token, where only the most likely candidates are kept and the rest are pruned at each decoding step, thus, retrieval fails if any token within the relevant DocID is falsely pruned. What's worse, during decoding, the model can only perceive preceding tokens in the DocID while being blind to subsequent ones, hence is prone to make such errors. To address this problem, we present a novel framework for generative retrieval, dubbed Term-Set Generation (TSGen). Instead of sequences, we use a set of terms as the DocID. The terms are selected based on learned weights from relevance signals, so that they concisely summarize the document's semantics and distinguish it from others. On top of the term-set DocID, we propose a permutation-invariant decoding algorithm, with which the term set can be generated in any permutation yet will always lead to the corresponding document. Remarkably, TSGen perceives all valid terms rather than only the preceding ones at each decoding step. Given the constant decoding space, it can make more reliable decisions due to the broader perspective. TSGen is also resilient to errors: the relevant DocID will not be falsely pruned as long as the decoded term belongs to it. Moreover, TSGen can explore the optimal decoding permutation of the term set on its own, which further improves the likelihood of generating the relevant DocID. Lastly, we design an iterative optimization procedure to incentivize the model to generate the relevant term set in its favorable permutation. We conduct extensive experiments on popular benchmarks of generative retrieval, which validate the effectiveness, the generalizability, the scalability, and the efficiency of TSGen.

Generative Retrieval Meets Multi-Graded Relevance

Listwise Generative Retrieval Models via a Sequential Learning Process

Enhancing Generative Retrieval with Reinforcement Learning from Relevance Feedback

A Survey of Generative Information Retrieval

Generative Retrieval as Multi-Vector Dense Retrieval

Hi-Gen: Generative Retrieval For Large-Scale Personalized E-commerce Search

Distillation Enhanced Generative Retrieval

From Matching to Generation: A Survey on Generative Information Retrieval

GRM: Generative Relevance Modeling Using Relevance-Aware Sample Estimation for Document Retrieval

Generative Dense Retrieval: Memory Can Be a Burden

Generative Retrieval Via Term Set Generation

Continual Learning for Generative Retrieval over Dynamic Corpora

Generative Retrieval as Dense Retrieval

Learning to Rank in Generative Retrieval

Exploring the Practicality of Generative Retrieval on Dynamic Corpora

ROGER: Ranking-oriented Generative Retrieval

Scalable and Effective Generative Information Retrieval

Learning to Tokenize for Generative Retrieval

Generative Retrieval with Few-shot Indexing

DR-RAG: Applying Dynamic Document Relevance to Retrieval-Augmented Generation for Question-Answering