Abstract:Retrieval-augmented generation (RAG) framework has shown promising potential in knowledge-intensive question answering (QA) by retrieving external corpus and generating based on augmented context. However, existing approaches only consider the query itself, neither specifying the retrieval preferences for the retrievers nor informing the generators of how to refer to the retrieved documents for the answers, which poses a significant challenge to the QA performance. To address these issues, we propose Rule-Guided Retrieval-Augmented Generation with LMs, which explicitly introduces symbolic rules as demonstrations for in-context learning (RuleRAG-ICL) to guide retrievers to retrieve logically related documents in the directions of rules and uniformly guide generators to generate answers attributed by the guidance of the same set of rules. Moreover, the combination of queries and rules can be further used as supervised fine-tuning data to update retrievers and generators (RuleRAG-FT) to achieve better rule-based instruction following capability, leading to retrieve more supportive results and generate more acceptable answers. To emphasize the attribution of rules, we construct five rule-aware QA benchmarks, including three temporal and two static scenarios, and equip RuleRAG with several kinds of retrievers and generators. Experiments demonstrate that training-free RuleRAG-ICL effectively improves the retrieval quality of +89.2% in Recall@10 scores and generation accuracy of +103.1% in exact match scores over standard RAG on average across the five benchmarks, and further fine-tuned RuleRAG-FT consistently yields more significant performance enhancement. Extensive analyses indicate that RuleRAG scales well with increasing numbers of retrieved documents and exhibits generalization ability for untrained rules.

Time-Sensitve Retrieval-Augmented Generation for Question Answering

MRAG: A Modular Retrieval Framework for Time-Sensitive Question Answering

Retriever-and-Memory: Towards Adaptive Note-Enhanced Retrieval-Augmented Generation

Optimizing Retrieval-Augmented Generation with Elasticsearch for Enhanced Question-Answering Systems

RetrievalQA: Assessing Adaptive Retrieval-Augmented Generation for Short-form Open-Domain Question Answering

DR-RAG: Applying Dynamic Document Relevance to Retrieval-Augmented Generation for Question-Answering

Retrieval-Augmented Generation for Domain-Specific Question Answering: A Case Study on Pittsburgh and CMU

Enhancing Temporal Sensitivity and Reasoning for Time-Sensitive Question Answering

LongRAG: A Dual-Perspective Retrieval-Augmented Generation Paradigm for Long-Context Question Answering

Layered Query Retrieval: an Adaptive Framework for Retrieval-Augmented Generation in Complex Question Answering for Large Language Models

A Multi-Source Retrieval Question Answering Framework Based on RAG

Searching for Best Practices in Retrieval-Augmented Generation

RichRAG: Crafting Rich Responses for Multi-faceted Queries in Retrieval-Augmented Generation

RuleRAG: Rule-guided retrieval-augmented generation with language models for question answering

Boosting Conversational Question Answering with Fine-Grained Retrieval-Augmentation and Self-Check

REAR: A Relevance-Aware Retrieval-Augmented Framework for Open-Domain Question Answering

Retrieval-Augmented Generation with Knowledge Graphs for Customer Service Question Answering

Retrieval-Augmented Generation for Natural Language Processing: A Survey

EfficientRAG: Efficient Retriever for Multi-Hop Question Answering

Enhancing Retrieval and Managing Retrieval: A Four-Module Synergy for Improved Quality and Efficiency in RAG Systems