Abstract:Knowledge Graph (KG) plays a crucial role in Medical Report Generation (MRG) because it reveals the relations among diseases and thus can be utilized to guide the generation process. However, constructing a comprehensive KG is labor-intensive and its applications on the MRG process are under-explored. In this study, we establish a complete KG on chest X-ray imaging that includes 137 types of diseases and abnormalities. Based on this KG, we find that the current MRG data sets exhibit a long-tailed problem in disease distribution. To mitigate this problem, we introduce a novel augmentation strategy that enhances the representation of disease types in the tail-end of the distribution. We further design a two-stage MRG approach, where a classifier is first trained to detect whether the input images exhibit any abnormalities. The classified images are then independently fed into two transformer-based generators, namely, ``disease-specific generator" and ``disease-free generator" to generate the corresponding reports. To enhance the clinical evaluation of whether the generated reports correctly describe the diseases appearing in the input image, we propose diverse sensitivity (DS), a new metric that checks whether generated diseases match ground truth and measures the diversity of all generated diseases. Results show that the proposed two-stage generation framework and augmentation strategies improve DS by a considerable margin, indicating a notable reduction in the long-tailed problem associated with under-represented diseases.

What problem does this paper attempt to address?

The problems that this paper attempts to solve mainly focus on the following aspects: 1. **Long - tail distribution problem**: In the existing medical report generation datasets, the distribution of disease types has a serious long - tail phenomenon, that is, the number of reports on a few common diseases is far more than that of rare diseases. This causes the model to be inclined to generate reports on common diseases while ignoring rare diseases, which affects the diversity and accuracy of the generated reports. 2. **Clinical relevance of disease identification and description**: The existing medical report generation methods mainly rely on N - gram matching degrees (such as BLEU scores) to evaluate the quality of the generated reports, but these indicators cannot well reflect the accuracy and clinical relevance of the disease descriptions in the generated reports. Therefore, a new evaluation indicator is needed to better measure the clinical value of the generated reports. 3. **Data imbalance problem**: In order to alleviate the long - tail distribution problem in the dataset, the paper proposes a new data augmentation strategy. By increasing the number of rare disease samples to balance the data distribution, the model's ability to generate reports on rare diseases is improved. 4. **Two - stage generation framework**: In order to process normal and abnormal images more effectively, the paper designs a two - stage generation framework. First, a classifier is used to detect whether the input image contains any abnormalities; then, according to the classification result, the corresponding generator ("disease - specific generator" or "disease - free generator") is selected to generate a report. This method helps to improve the pertinence and accuracy of the generated reports. By constructing a comprehensive knowledge graph, proposing a new data augmentation strategy, and designing a two - stage generation framework, this paper aims to improve the accuracy and diversity of disease descriptions in the medical report generation task, especially the performance in dealing with rare diseases. At the same time, the paper also introduces a new evaluation indicator - Diverse Sensitivity (DS) - to better evaluate the clinical relevance of the generated reports.

Rethinking Medical Report Generation: Disease Revealing Enhancement with Knowledge Graph

Improving Medical X-ray Report Generation by Using Knowledge Graph

A Doctors Behavior Aware and Domain Knowledge Driven Model for Medical Report Generation

KARGEN: Knowledge-enhanced Automated Radiology Report Generation Using Large Language Models

Knowledge-Driven Encode, Retrieve, Paraphrase for Medical Image Report Generation

When Radiology Report Generation Meets Knowledge Graph

Enhanced Knowledge Injection for Radiology Report Generation

Knowledge Matters: Radiology Report Generation with General and Specific Knowledge

DKA-RG: Disease-Knowledge-Enhanced Fine-Grained Image–Text Alignment for Automatic Radiology Report Generation

Medical Report Generation Via Multimodal Spatio-Temporal Fusion

Auto-Encoding Knowledge Graph for Unsupervised Medical Report Generation

Few-Shot Radiology Report Generation via Knowledge Transfer and Multi-modal Alignment.

Uncovering Knowledge Gaps in Radiology Report Generation Models through Knowledge Graphs

Improving Radiology Report Generation with Multi-Grained Abnormality Prediction

Knowledge matters: Chest radiology report generation with general and specific knowledge

Automated Generation of Accurate \& Fluent Medical X-ray Reports

Vision-knowledge fusion model for multi-domain medical report generation

Medical Report Generation Is A Multi-label Classification Problem

Radiology Report Generation with a Learned Knowledge Base and Multi-Modal Alignment

Bridging the Gap: Cross-modal Knowledge Driven Network for Radiology Report Generation

Rethinking Radiology Report Generation Via Causal Reasoning and Counterfactual Augmentation.