Abstract:Large Vision Language Models (LVLMs) are increasingly integral to healthcare applications, including medical visual question answering and imaging report generation. While these models inherit the robust capabilities of foundational Large Language Models (LLMs), they also inherit susceptibility to hallucinations-a significant concern in high-stakes medical contexts where the margin for error is minimal. However, currently, there are no dedicated methods or benchmarks for hallucination detection and evaluation in the medical field. To bridge this gap, we introduce Med-HallMark, the first benchmark specifically designed for hallucination detection and evaluation within the medical multimodal domain. This benchmark provides multi-tasking hallucination support, multifaceted hallucination data, and hierarchical hallucination categorization. Furthermore, we propose the MediHall Score, a new medical evaluative metric designed to assess LVLMs' hallucinations through a hierarchical scoring system that considers the severity and type of hallucination, thereby enabling a granular assessment of potential clinical impacts. We also present MediHallDetector, a novel Medical LVLM engineered for precise hallucination detection, which employs multitask training for hallucination detection. Through extensive experimental evaluations, we establish baselines for popular LVLMs using our benchmark. The findings indicate that MediHall Score provides a more nuanced understanding of hallucination impacts compared to traditional metrics and demonstrate the enhanced performance of MediHallDetector. We hope this work can significantly improve the reliability of LVLMs in medical applications. All resources of this work will be released soon.

PhD: A Prompted Visual Hallucination Evaluation Dataset

PhD: A ChatGPT-Prompted Visual hallucination Evaluation Dataset

HallusionBench: An Advanced Diagnostic Suite for Entangled Language Hallucination and Visual Illusion in Large Vision-Language Models

AutoHallusion: Automatic Generation of Hallucination Benchmarks for Vision-Language Models

Hal-Eval: A Universal and Fine-grained Hallucination Evaluation Framework for Large Vision Language Models

Evaluating Object Hallucination in Large Vision-Language Models

Evaluating the Quality of Hallucination Benchmarks for Large Vision-Language Models

Evaluation and Analysis of Hallucination in Large Vision-Language Models

A Survey on Hallucination in Large Vision-Language Models

Unified Triplet-Level Hallucination Evaluation for Large Vision-Language Models

MedVH: Towards Systematic Evaluation of Hallucination for Large Vision Language Models in the Medical Context

HalluDial: A Large-Scale Benchmark for Automatic Dialogue-Level Hallucination Evaluation

VisDiaHalBench: A Visual Dialogue Benchmark For Diagnosing Hallucination in Large Vision-Language Models

Visual Hallucination: Definition, Quantification, and Prescriptive Remediations

VideoHallucer: Evaluating Intrinsic and Extrinsic Hallucinations in Large Video-Language Models

A Survey of Hallucination in Large Visual Language Models

HalluciDoctor: Mitigating Hallucinatory Toxicity in Visual Instruction Data

The Dawn After the Dark: An Empirical Study on Factuality Hallucination in Large Language Models

Detecting and Evaluating Medical Hallucinations in Large Vision Language Models