Abstract:The rapidly developing Large Vision Language Models (LVLMs) have shown notable capabilities on a range of multi-modal tasks, but still face the hallucination phenomena where the generated texts do not align with the given contexts, significantly restricting the usages of LVLMs. Most previous work detects and mitigates hallucination at the coarse-grained level or requires expensive annotation (e.g., labeling by proprietary models or human experts). To address these issues, we propose detecting and mitigating hallucinations in LVLMs via fine-grained AI feedback. The basic idea is that we generate a small-size sentence-level hallucination annotation dataset by proprietary models, whereby we train a hallucination detection model which can perform sentence-level hallucination detection, covering primary hallucination types (i.e., object, attribute, and relationship). Then, we propose a detect-then-rewrite pipeline to automatically construct preference dataset for training hallucination mitigating model. Furthermore, we propose differentiating the severity of hallucinations, and introducing a Hallucination Severity-Aware Direct Preference Optimization (HSA-DPO) for mitigating hallucination in LVLMs by incorporating the severity of hallucinations into preference learning. Extensive experiments demonstrate the effectiveness of our method.

What problem does this paper attempt to address?

This paper attempts to address the hallucination phenomenon in large - scale visual - language models (LVLMs), that is, the problem of generated text being inconsistent with the given context. Specifically, the paper focuses on the following aspects: 1. **Hallucination Detection**: Most of the existing research detects hallucinations at a coarse - grained level or requires expensive manual annotation. This paper proposes a method based on fine - grained AI feedback to detect hallucinations, which can perform sentence - level hallucination detection and cover the main hallucination types (objects, attributes, and relationships). 2. **Hallucination Mitigation**: The existing mitigation methods either require additional training costs or sacrifice inference speed. This paper proposes a pipeline for automatically constructing preference datasets (detect - then - rewrite pipeline) and mitigates hallucinations by introducing Hallucination Severity - Aware Direct Preference Optimization (HSA - DPO) to prioritize critical hallucinations. ### Main Contributions of the Paper: 1. **Fine - grained AI Feedback**: For the first time, a powerful closed - source model is used to generate fine - grained AI feedback for hallucination detection and mitigation, which is not only effective but also reduces the need for manual annotation. 2. **Automatic Preference Dataset Construction Pipeline**: The preference dataset is automatically constructed by two expert models, further reducing the annotation cost and facilitating the expansion of the preference scale. 3. **Hallucination Severity - Aware Direct Preference Optimization (HSA - DPO)**: A method for distinguishing the severity of hallucinations is proposed and incorporated into preference learning to prioritize the mitigation of critical hallucinations. 4. **Extensive Experimental Verification**: Extensive experiments have been carried out on multiple multi - modal hallucination detection and mitigation benchmarks, and the results show that the proposed method has achieved significant results in both hallucination detection and mitigation. ### Experimental Results: - **Hallucination Detection**: On the MHaluBench benchmark, the proposed detection model (H - DER) outperforms GPT - 4V and Gemini in terms of average performance. Specifically, at the claim level, the Mac. F1 score of H - DER is 4.7% higher than that of UNIHD and 8.1% higher than that of GPT - 4V Self - Check 2 - shot. On the fine - grained detection dataset, H - DER achieves an F1 score of 88.2% in the binary classification task and an accuracy of 74.3% in the fine - grained classification task. - **Hallucination Mitigation**: On the Object HalBench benchmark, HSA - DPO achieves state - of - the - art results and even outperforms the leading LVLMs GPT - 4V. On the AMBER benchmark, HSA - DPO reduces the hallucination rate by 36.1%, and on the Object HalBench, it reduces CHAIRs by 76.3%. These results indicate that fine - grained AI feedback and HSA - DPO have significant advantages in improving the reliability and accuracy of LVLMs.

Detecting and Mitigating Hallucination in Large Vision Language Models via Fine-Grained AI Feedback

A Survey on Hallucination in Large Vision-Language Models

Evaluation and Analysis of Hallucination in Large Vision-Language Models

Alleviating Hallucination in Large Vision-Language Models with Active Retrieval Augmentation

A Survey of Hallucination in Large Visual Language Models

Detecting and Evaluating Medical Hallucinations in Large Vision Language Models

Analyzing and Mitigating Object Hallucination in Large Vision-Language Models

HELPD: Mitigating Hallucination of LVLMs by Hierarchical Feedback Learning with Vision-enhanced Penalty Decoding

Mitigating Multilingual Hallucination in Large Vision-Language Models

Cracking the Code of Hallucination in LVLMs with Vision-aware Head Divergence

Hal-Eval: A Universal and Fine-grained Hallucination Evaluation Framework for Large Vision Language Models

A Unified Hallucination Mitigation Framework for Large Vision-Language Models

Towards Mitigating Hallucination in Large Language Models via Self-Reflection

Reducing Hallucinations in Vision-Language Models via Latent Space Steering

Hallucination of Multimodal Large Language Models: A Survey

Alleviating Hallucinations in Large Vision-Language Models through Hallucination-Induced Optimization

Devils in Middle Layers of Large Vision-Language Models: Interpreting, Detecting and Mitigating Object Hallucinations via Attention Lens