Abstract:Retrieval-augmented large language models (LLMs) leverage relevant content retrieved by information retrieval systems to generate correct responses, aiming to alleviate the hallucination problem. However, existing retriever-responder methods typically append relevant documents to the prompt of LLMs to perform text generation tasks without considering the interaction of fine-grained structural semantics between the retrieved documents and the LLMs. This issue is particularly important for accurate response generation as LLMs tend to ``lose in the middle'' when dealing with input prompts augmented with lengthy documents. In this work, we propose a new pipeline named ``Reinforced Retriever-Reorder-Responder'' (R$^4$) to learn document orderings for retrieval-augmented LLMs, thereby further enhancing their generation abilities while the large numbers of parameters of LLMs remain frozen. The reordering learning process is divided into two steps according to the quality of the generated responses: document order adjustment and document representation enhancement. Specifically, document order adjustment aims to organize retrieved document orderings into beginning, middle, and end positions based on graph attention learning, which maximizes the reinforced reward of response quality. Document representation enhancement further refines the representations of retrieved documents for responses of poor quality via document-level gradient adversarial learning. Extensive experiments demonstrate that our proposed pipeline achieves better factual question-answering performance on knowledge-intensive tasks compared to strong baselines across various public datasets. The source codes and trained models will be released upon paper acceptance.

A Large Language Model-based Framework for Semi-Structured Tender Document Retrieval-Augmented Generation

Retrieval-Generation Synergy Augmented Large Language Models

Evaluating Self-Generated Documents for Enhancing Retrieval-Augmented Generation with Large Language Models

Legal Documents Drafting with Fine-Tuned Pre-Trained Large Language Model

Enhancing Retrieval-Augmented Large Language Models with Iterative Retrieval-Generation Synergy

A Method for Parsing and Vectorization of Semi-structured Data used in Retrieval Augmented Generation

Impact of Large Language Models on Generating Software Specifications

DocReLM: Mastering Document Retrieval with Language Model

LLM Based Multi-Agent Generation of Semi-structured Documents from Semantic Templates in the Public Administration Domain

Retrieval-Augmented Generation for Large Language Models: A Survey

LLM-Augmented Retrieval: Enhancing Retrieval Models Through Language Models and Doc-Level Embedding

R4: Reinforced Retriever-Reorder-Responder for Retrieval-Augmented Large Language Models

RETA-LLM: A Retrieval-Augmented Large Language Model Toolkit

Generative Retrieval with Large Language Models

Think-then-Act: A Dual-Angle Evaluated Retrieval-Augmented Generation

A + B: A General Generator-Reader Framework for Optimizing LLMs to Unleash Synergy Potential

Development and Testing of Retrieval Augmented Generation in Large Language Models -- A Case Study Report

Redefining Information Retrieval of Structured Database via Large Language Models

Large Language Models Empower Personalized Valuation in Auction

Enhancing Legal Document Retrieval: A Multi-Phase Approach with Large Language Models

Large Language Model Supply Chain: A Research Agenda