Abstract:Retrieval-augmented large language models (LLMs) leverage relevant content retrieved by information retrieval systems to generate correct responses, aiming to alleviate the hallucination problem. However, existing retriever-responder methods typically append relevant documents to the prompt of LLMs to perform text generation tasks without considering the interaction of fine-grained structural semantics between the retrieved documents and the LLMs. This issue is particularly important for accurate response generation as LLMs tend to ``lose in the middle'' when dealing with input prompts augmented with lengthy documents. In this work, we propose a new pipeline named ``Reinforced Retriever-Reorder-Responder'' (R$^4$) to learn document orderings for retrieval-augmented LLMs, thereby further enhancing their generation abilities while the large numbers of parameters of LLMs remain frozen. The reordering learning process is divided into two steps according to the quality of the generated responses: document order adjustment and document representation enhancement. Specifically, document order adjustment aims to organize retrieved document orderings into beginning, middle, and end positions based on graph attention learning, which maximizes the reinforced reward of response quality. Document representation enhancement further refines the representations of retrieved documents for responses of poor quality via document-level gradient adversarial learning. Extensive experiments demonstrate that our proposed pipeline achieves better factual question-answering performance on knowledge-intensive tasks compared to strong baselines across various public datasets. The source codes and trained models will be released upon paper acceptance.

Learning a Replacement Model for Query Segmentation with Consistency in Search Logs.

Learning with Both Unlabeled Data and Query Logs for Image Search.

An error driven approach to query segmentation

Query Segmentation for Relevance Ranking in Web Search

Learning Equivariant Segmentation with Instance-Unique Querying

SegLLM: Multi-round Reasoning Segmentation

RQ-RAG: Learning to Refine Queries for Retrieval Augmented Generation

Leveraging Task-Specific Knowledge from LLM for Semi-Supervised 3D Medical Image Segmentation

Video Object Segmentation via Global Consistency Aware Query Strategy.

LaSagnA: Language-based Segmentation Assistant for Complex Queries

Exploring Segment Representations for Neural Segmentation Models

LMSeg: Unleashing the Power of Large-Scale Models for Open-Vocabulary Semantic Segmentation

MRL-Seg: Overcoming Imbalance in Medical Image Segmentation With Multi-Step Reinforcement Learning

LaGR-SEQ: Language-Guided Reinforcement Learning with Sample-Efficient Querying

ConQueR: Contextualized Query Reduction using Search Logs

Filling Memory Gaps: Enhancing Continual Semantic Parsing via SQL Syntax Variance-Guided LLMs without Real Data Replay

See, Say, and Segment: Teaching LMMs to Overcome False Premises

A Masked Segmental Language Model for Unsupervised Natural Language Segmentation

Meta-Chunking: Learning Efficient Text Segmentation via Logical Perception

R4: Reinforced Retriever-Reorder-Responder for Retrieval-Augmented Large Language Models

Fully and Weakly Supervised Referring Expression Segmentation with End-to-End Learning