Abstract:Large Language Models (LLMs) have demonstrated remarkable generation capabilities but often struggle to access up-to-date information, which can lead to hallucinations. Retrieval-Augmented Generation (RAG) addresses this issue by incorporating knowledge from external databases, enabling more accurate and relevant responses. Due to the context window constraints of LLMs, it is impractical to input the entire external database context directly into the model. Instead, only the most relevant information, referred to as chunks, is selectively retrieved. However, current RAG research faces three key challenges. First, existing solutions often select each chunk independently, overlooking potential correlations among them. Second, in practice the utility of chunks is non-monotonic, meaning that adding more chunks can decrease overall utility. Traditional methods emphasize maximizing the number of included chunks, which can inadvertently compromise performance. Third, each type of user query possesses unique characteristics that require tailored handling, an aspect that current approaches do not fully consider. To overcome these challenges, we propose a cost constrained retrieval optimization system CORAG for retrieval-augmented generation. We employ a Monte Carlo Tree Search (MCTS) based policy framework to find optimal chunk combinations sequentially, allowing for a comprehensive consideration of correlations among chunks. Additionally, rather than viewing budget exhaustion as a termination condition, we integrate budget constraints into the optimization of chunk combinations, effectively addressing the non-monotonicity of chunk utility.

Introducing a new hyper-parameter for RAG: Context Window Utilization

RAGGED: Towards Informed Design of Retrieval Augmented Generation Systems

In Defense of RAG in the Era of Long-Context Language Models

Better RAG using Relevant Information Gain

Lighter And Better: Towards Flexible Context Adaptation For Retrieval Augmented Generation

Context Embeddings for Efficient Answer Generation in RAG

Context Awareness Gate For Retrieval Augmented Generation

Context Tuning for Retrieval Augmented Generation

xRAG: Extreme Context Compression for Retrieval-augmented Generation with One Token

Quantifying reliance on external information over parametric knowledge during Retrieval Augmented Generation (RAG) using mechanistic analysis

Long-Context LLMs Meet RAG: Overcoming Challenges for Long Inputs in RAG

RE-RAG: Improving Open-Domain QA Performance and Interpretability with Relevance Estimator in Retrieval-Augmented Generation

Accelerating Inference of Retrieval-Augmented Generation via Sparse Context Selection

Optimization of Retrieval-Augmented Generation Context with Outlier Detection

Adaptive Retrieval-Augmented Generation for Conversational Systems

Sufficient Context: A New Lens on Retrieval Augmented Generation Systems

Long Context RAG Performance of Large Language Models

CORAG: A Cost-Constrained Retrieval Optimization System for Retrieval-Augmented Generation

Dynamic Contexts for Generating Suggestion Questions in RAG Based Conversational Systems

LongRAG: A Dual-Perspective Retrieval-Augmented Generation Paradigm for Long-Context Question Answering