Abstract:Retriever Augmented Generation (RAG) systems have become pivotal in enhancing the capabilities of language models by incorporating external knowledge retrieval mechanisms. However, a significant challenge in deploying these systems in industry applications is the detection and mitigation of hallucinations: instances where the model generates information that is not grounded in the retrieved context. Addressing this issue is crucial for ensuring the reliability and accuracy of responses generated by large language models (LLMs) in diverse industry settings. Current hallucination detection techniques fail to deliver accuracy, low latency, and low cost simultaneously. We introduce Luna: a DeBERTA-large (440M) encoder, finetuned for hallucination detection in RAG settings. We demonstrate that Luna outperforms GPT-3.5 and commercial evaluation frameworks on the hallucination detection task, with 97% and 91% reduction in cost and latency, respectively. Luna is lightweight and generalizes across multiple industry verticals and out-of-domain data, making it an ideal candidate for industry LLM applications.

What problem does this paper attempt to address?

The main problem that this paper attempts to solve is the hallucinations in the responses generated by large - language models (LLMs) in practical applications, that is, the information generated by the model is inconsistent or untrue with the retrieved context. Specifically, the paper focuses on how to detect these hallucinations efficiently and at low cost, especially in the case of using retrieval - augmented generation (RAG) systems. Current hallucination - detection techniques are either not highly accurate or have high latency and are expensive. Therefore, the paper proposes a new model named Luna, aiming to provide high - precision hallucination - detection capabilities at a lower cost and with lower latency. ### Background of the Paper - **Challenges of RAG Systems**: RAG systems enhance the capabilities of language models by combining external knowledge retrieval mechanisms, but in practical applications, the model may still generate information inconsistent with the retrieved context, that is, hallucinations. - **Deficiencies of Existing Solutions**: Existing hallucination - detection methods are difficult to achieve high precision, low latency and low cost simultaneously. ### Features of the Luna Model - **Model Architecture**: Luna is fine - tuned based on the DeBERTa - large (440 million parameters) encoder and is specifically used to detect hallucinations in the RAG setting. - **Performance Advantages**: Luna performs excellently in the hallucination - detection task. Compared with GPT - 3.5 and commercial evaluation frameworks, its cost is reduced by 97% and its latency is reduced by 91%. - **Generalization Ability**: Luna can generalize across multiple industry sectors and out - of - domain data and is suitable for various practical application scenarios. ### Main Contributions 1. **Proposing the Luna Model**: A lightweight DeBERTa - large encoder, fine - tuned with carefully selected real - world RAG data, can effectively detect hallucinations in multiple industry sectors. 2. **Long - Context Processing**: Aiming at the under - studied challenge of long - context RAG evaluation, a new solution is proposed, which can detect hallucinations in long contexts with high precision. 3. **Performance Optimization**: Luna has carried out multiple optimizations during deployment and can process up to 16,000 input tokens on an NVIDIA L4 GPU with a latency of less than 1 second. ### Experimental Results - **Benchmark Tests**: Luna outperforms all prompt - based methods on the RAGTruth dataset and is comparable to the GPT - 3.5 evaluator in data - to - text writing tasks. - **Cross - Domain Generalization**: Luna performs excellently on RAG QA test sets in different domains, especially in long - context processing, and its performance is significantly better than other baseline models. ### Conclusion The Luna model provides an efficient, low - cost and high - precision hallucination - detection solution, which is especially suitable for application in actual production environments. Through local GPU deployment, Luna also ensures the privacy and security of data.

Luna: An Evaluation Foundation Model to Catch Language Model Hallucinations with High Accuracy and Low Cost

Retrieve Only When It Needs: Adaptive Retrieval Augmentation for Hallucination Mitigation in Large Language Models

HaluEval: A Large-Scale Hallucination Evaluation Benchmark for Large Language Models

Detecting Hallucinations in Large Language Model Generation: A Token Probability Approach

Lynx: An Open Source Hallucination Evaluation Model

Hallucination Detection and Hallucination Mitigation: An Investigation

Cost-Effective Hallucination Detection for LLMs

Analyzing and Mitigating Object Hallucination in Large Vision-Language Models

Alleviating Hallucination in Large Vision-Language Models with Active Retrieval Augmentation

Small Agent Can Also Rock! Empowering Small Language Models as Hallucination Detector

Fine-grained Hallucination Detection and Editing for Language Models

Enhancing LLM Factual Accuracy with RAG to Counter Hallucinations: A Case Study on Domain-Specific Queries in Private Knowledge-Bases

HaluEval-Wild: Evaluating Hallucinations of Language Models in the Wild

A Stitch in Time Saves Nine: Detecting and Mitigating Hallucinations of LLMs by Validating Low-Confidence Generation

In-Context Learning for Scalable and Online Hallucination Detection in RAGS

LRP4RAG: Detecting Hallucinations in Retrieval-Augmented Generation via Layer-wise Relevance Propagation

ReDeEP: Detecting Hallucination in Retrieval-Augmented Generation via Mechanistic Interpretability

Hallucination Detection in LLMs: Fast and Memory-Efficient Fine-Tuned Models

MedHalu: Hallucinations in Responses to Healthcare Queries by Large Language Models