Abstract:Chest radiology imaging plays a crucial role in the early screening, diagnosis, and treatment of chest diseases. The accurate interpretation of radiological images and the automatic generation of radiology reports not only save the doctor's time but also mitigate the risk of errors in diagnosis. The core objective of automatic radiology report generation is to achieve precise mapping of visual features and lesion descriptions at multi-scale and fine-grained levels. Existing methods typically combine global visual features and textual features to generate radiology reports. However, these approaches may ignore the key lesion areas and lack sensitivity to crucial lesion location information. Furthermore, achieving multi-scale characterization and fine-grained alignment of medical visual features and report text features proves challenging, leading to a reduction in the quality of radiology report generation. Addressing these issues, we propose a method for chest radiology report generation based on cross-modal multi-scale feature fusion. First, an auxiliary labeling module is designed to guide the model to focus on the lesion region of the radiological image. Second, a channel attention network is employed to enhance the characterization of location information and disease features. Finally, a cross-modal features fusion module is constructed by combining memory matrices, facilitating fine-grained alignment between multi-scale visual features and reporting text features on corresponding scales. The proposed method is experimentally evaluated on two publicly available radiological image datasets. The results demonstrate superior performance based on BLEU and ROUGE metrics compared to existing methods. Particularly, there are improvements of 4.8% in the ROUGE metric and 9.4% in the METEOR metric on the IU X-Ray dataset. Moreover, there is a 7.4% enhancement in BLEU-1 and a 7.6% improvement in the BLEU-2 on the MIMIC-CXR dataset.

Diversity-Preserving Chest Radiographs Generation from Reports in One Stage

An Inclusive Task-Aware Framework for Radiology Report Generation

Act Like a Radiologist: Radiology Report Generation across Anatomical Regions

Clinically Accurate Chest X-Ray Report Generation

XRayGAN: Consistency-preserving Generation of X-ray Images from Radiology Reports

Clinically Coherent Radiology Report Generation with Imbalanced Chest X-rays.

Automated Radiological Report Generation For Chest X-Rays With Weakly-Supervised End-to-End Deep Learning

Medical Report Generation Via Multimodal Spatio-Temporal Fusion

Automatic Report Generation for Chest X-Ray Images Via Adversarial Reinforcement Learning.

A Survey of Deep Learning-based Radiology Report Generation Using Multimodal Data

Automated Chest X-Ray Report Generator Using Multi-Model Deep Learning Approach

Chest X-ray Report Generation through Fine-Grained Label Learning

Dynamic Multi-Domain Knowledge Networks for Chest X-ray Report Generation

Chest radiology report generation based on cross-modal multi-scale feature fusion

Representative Image Feature Extraction via Contrastive Learning Pretraining for Chest X-ray Report Generation

Deep Learning for Report Generation on Chest X-ray Images

Visualizing chest X-ray dataset biases using GANs

Radiology Report Generation via Structured Knowledge-Enhanced Multi-modal Attention and Contrastive Learning.

Automated Generation of Accurate \& Fluent Medical X-ray Reports

Generation of Radiology Findings in Chest X-Ray by Leveraging Collaborative Knowledge

Knowledge matters: Chest radiology report generation with general and specific knowledge