Mix-of-Granularity: Optimize the Chunking Granularity for Retrieval-Augmented Generation

Zijie Zhong,Hanwen Liu,Xiaoya Cui,Xiaofan Zhang,Zengchang Qin

2024-06-01

Abstract:Integrating information from different reference data sources is a major challenge for Retrieval-Augmented Generation (RAG) systems because each knowledge source adopts a unique data structure and follows different conventions. Retrieving from multiple knowledge sources with one fixed strategy usually leads to under-exploitation of information. To mitigate this drawback, inspired by Mix-of-Expert, we introduce Mix-of-Granularity (MoG), a method that dynamically determines the optimal granularity of a knowledge database based on input queries using a router. The router is efficiently trained with a newly proposed loss function employing soft labels. We further extend MoG to Mix-of-Granularity-Graph (MoGG), where reference documents are pre-processed into graphs, enabling the retrieval of relevant information from distantly situated chunks. Extensive experiments demonstrate that both MoG and MoGG effectively predict optimal granularity levels, significantly enhancing the performance of the RAG system in downstream tasks. The code of both MoG and MoGG will be made public.

Machine Learning,Artificial Intelligence

What problem does this paper attempt to address?

The paper aims to address the main challenge of integrating information from different reference data sources in Retrieval-Augmented Generation (RAG) systems. Specifically, since each knowledge source adopts a unique data structure and follows different specifications, using a fixed strategy to retrieve information from multiple knowledge sources often leads to underutilization of information. To mitigate this shortcoming, the authors are inspired by the "Mix-of-Experts" approach and propose a method called "Mix-of-Granularity" (MoG), which can dynamically determine the optimal granularity of the knowledge database based on the input query. Furthermore, the authors extend MoG to "Mix-of-Granularity-Graph" (MoGG), where reference documents are preprocessed into graph form, allowing relevant information to be retrieved from distantly separated blocks. The core contributions of the paper include: 1. Proposing MoG, which dynamically selects fragments of different granularity levels through a router, achieving a balance between coverage and relevance in the retrieval phase of RAG systems. 2. Extending MoG to MoGG by reorganizing reference documents into graph form, thereby further improving the quality of information retrieval when relevant information is scattered across different parts of the knowledge base. 3. Introducing a loss function using soft labels to overcome challenges related to top-k selection during training.

Mix-of-Granularity: Optimize the Chunking Granularity for Retrieval-Augmented Generation

CORAG: A Cost-Constrained Retrieval Optimization System for Retrieval-Augmented Generation

$\texttt{MixGR}$: Enhancing Retriever Generalization for Scientific Domain through Complementary Granularity

RichRAG: Crafting Rich Responses for Multi-faceted Queries in Retrieval-Augmented Generation

GRAG: Graph Retrieval-Augmented Generation

Meta-Chunking: Learning Efficient Text Segmentation via Logical Perception

Towards Multi-Source Retrieval-Augmented Generation via Synergizing Reasoning and Preference-Driven Retrieval

Multi-Source Knowledge Pruning for Retrieval-Augmented Generation: A Benchmark and Empirical Study

Simple is Effective: The Roles of Graphs and Large Language Models in Knowledge-Graph-Based Retrieval-Augmented Generation

DuetRAG: Collaborative Retrieval-Augmented Generation

MemoRAG: Moving towards Next-Gen RAG Via Memory-Inspired Knowledge Discovery

MBA-RAG: a Bandit Approach for Adaptive Retrieval-Augmented Generation through Question Complexity

WeKnow-RAG: An Adaptive Approach for Retrieval-Augmented Generation Integrating Web Search and Knowledge Graphs

RAG-DDR: Optimizing Retrieval-Augmented Generation Using Differentiable Data Rewards

DR-RAG: Applying Dynamic Document Relevance to Retrieval-Augmented Generation for Question-Answering

LightRAG: Simple and Fast Retrieval-Augmented Generation

Speculative RAG: Enhancing Retrieval Augmented Generation through Drafting

CoFE-RAG: A Comprehensive Full-chain Evaluation Framework for Retrieval-Augmented Generation with Enhanced Data Diversity

FunnelRAG: A Coarse-to-Fine Progressive Retrieval Paradigm for RAG

Retriever-and-Memory: Towards Adaptive Note-Enhanced Retrieval-Augmented Generation

LLMs Know What They Need: Leveraging a Missing Information Guided Framework to Empower Retrieval-Augmented Generation